Sep 09 21:08:04.848348 ip-10-0-128-60 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 09 21:08:04.848363 ip-10-0-128-60 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 09 21:08:04.848371 ip-10-0-128-60 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 09 21:08:04.848871 ip-10-0-128-60 systemd[1]: Failed to start Kubernetes Kubelet. Sep 09 21:08:14.889677 ip-10-0-128-60 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 09 21:08:14.889699 ip-10-0-128-60 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 66ad869917314f41b7828a39a5e71477 -- Sep 09 21:09:57.370766 ip-10-0-128-60 systemd[1]: Starting Kubernetes Kubelet... Sep 09 21:09:57.751291 ip-10-0-128-60 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:09:57.751291 ip-10-0-128-60 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 21:09:57.751291 ip-10-0-128-60 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:09:57.751291 ip-10-0-128-60 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 21:09:57.751291 ip-10-0-128-60 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:09:57.752006 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.751370 2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 21:09:57.754258 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754240 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:57.754258 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754257 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754261 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754265 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754268 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754273 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754277 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754281 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754284 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754287 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754289 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754292 2550 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754295 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754297 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754300 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754302 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754305 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754307 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754310 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754314 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:57.754333 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754332 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754337 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754340 2550 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754343 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754346 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754349 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754352 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754355 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754358 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754361 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754363 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754366 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754370 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754373 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754376 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754378 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754381 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754383 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754385 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:57.754782 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754388 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754391 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754394 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754396 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754398 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754401 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754405 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754408 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754411 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754413 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754416 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754418 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754421 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754423 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754426 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754430 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754433 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754436 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754439 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:57.755230 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754441 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754444 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754446 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754449 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754451 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754453 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754456 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754459 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754461 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754464 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754466 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754469 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754471 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754474 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754477 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754480 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754483 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754486 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754488 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754491 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754493 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:57.755712 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754496 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754499 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754501 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754504 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754506 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754508 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754511 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754514 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754518 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754520 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754523 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754525 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754528 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754531 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754533 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754536 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754539 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754541 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754543 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:57.756566 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.754546 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756876 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756891 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756895 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756898 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756902 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756905 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756907 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756910 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756913 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756915 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756918 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756920 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756923 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756925 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756928 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756931 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756933 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756936 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:57.757053 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756938 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756941 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756943 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756946 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756948 2550 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756951 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756953 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756956 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756958 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756961 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756963 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756967 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756969 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756971 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756974 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756977 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756980 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756982 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756985 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756989 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:57.757552 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756992 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756995 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.756998 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757000 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757003 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757007 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757011 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757014 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757017 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757020 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757022 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757026 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757030 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757033 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757036 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757038 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757041 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757043 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757046 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:57.758041 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757048 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757051 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757054 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757056 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757059 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757062 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757064 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757067 2550 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757071 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757074 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757076 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757080 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757083 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757085 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757088 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757091 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757093 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757096 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757098 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:57.758521 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757101 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757103 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757106 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757108 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757111 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757113 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757116 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757119 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757122 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757124 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757127 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757129 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757132 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757134 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757137 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757139 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757141 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757144 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757146 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757149 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:57.759012 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757151 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757155 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757158 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757571 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757576 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757581 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757585 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757588 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757591 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757594 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757597 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757602 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757606 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757610 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757614 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757617 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757621 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757625 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757630 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:57.759510 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757635 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757638 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757641 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757643 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757646 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757648 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757651 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757654 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757657 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757659 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757662 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757664 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757674 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757677 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757680 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757683 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757685 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757688 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757691 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757693 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:57.759963 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757697 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757700 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757703 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757706 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757708 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757711 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757714 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757716 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757719 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757721 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757724 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757727 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757729 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757733 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757735 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757738 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757740 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757743 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757745 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757748 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:57.760476 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757750 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757753 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757755 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757757 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757760 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757762 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757765 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757768 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757770 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757775 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757779 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757782 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757785 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757788 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757790 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757793 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757795 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757798 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757800 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:57.760949 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757803 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757805 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757808 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757810 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757813 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757815 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757818 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757821 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757824 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757826 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757829 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757831 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757833 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757836 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757839 2550 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757841 2550 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757843 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757846 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757848 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757851 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:57.761410 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757854 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757856 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757858 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.757861 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.757938 2550 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.757985 2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.757993 2550 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.757997 2550 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758001 2550 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758005 2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758009 2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758013 2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758016 2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758019 2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758023 2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758026 2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758030 2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758033 2550 flags.go:64] FLAG: --cgroup-root="" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758036 2550 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758040 2550 flags.go:64] FLAG: --client-ca-file="" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758044 2550 flags.go:64] FLAG: --cloud-provider="external" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758047 2550 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758051 2550 flags.go:64] FLAG: --cluster-domain="" Sep 09 21:09:57.761893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758054 2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758057 2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758060 2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758063 2550 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758071 2550 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758074 2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758077 2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758082 2550 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758085 2550 flags.go:64] FLAG: --contention-profiling="false" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758088 2550 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758091 2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758094 2550 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758097 2550 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758101 2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758105 2550 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758107 2550 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758110 2550 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758113 2550 flags.go:64] FLAG: --enable-server="true" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758116 2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758120 2550 flags.go:64] FLAG: --event-burst="100" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758123 2550 flags.go:64] FLAG: --event-qps="50" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758126 2550 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758128 2550 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758131 2550 flags.go:64] FLAG: --eviction-hard="" Sep 09 21:09:57.762448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758135 2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758138 2550 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758141 2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758144 2550 flags.go:64] FLAG: --eviction-soft="" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758147 2550 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758149 2550 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758153 2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758155 2550 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758158 2550 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758161 2550 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758164 2550 flags.go:64] FLAG: --feature-gates="" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758168 2550 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758170 2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758173 2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758176 2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758180 2550 flags.go:64] FLAG: --healthz-port="10248" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758183 2550 flags.go:64] FLAG: --help="false" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758186 2550 flags.go:64] FLAG: --hostname-override="ip-10-0-128-60.ec2.internal" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758189 2550 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758191 2550 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758194 2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758197 2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758200 2550 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758203 2550 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 21:09:57.763013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758206 2550 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758209 2550 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758212 2550 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758214 2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758217 2550 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758220 2550 flags.go:64] FLAG: --kube-reserved="" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758223 2550 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758225 2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758228 2550 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758231 2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758234 2550 flags.go:64] FLAG: --lock-file="" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758237 2550 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758240 2550 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758242 2550 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758248 2550 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758251 2550 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758254 2550 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758257 2550 flags.go:64] FLAG: --logging-format="text" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758259 2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758262 2550 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758265 2550 flags.go:64] FLAG: --manifest-url="" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758267 2550 flags.go:64] FLAG: --manifest-url-header="" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758272 2550 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758276 2550 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758280 2550 flags.go:64] FLAG: --max-pods="110" Sep 09 21:09:57.763648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758283 2550 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758286 2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758289 2550 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758292 2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758294 2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758297 2550 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758300 2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758307 2550 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758311 2550 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758313 2550 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758332 2550 flags.go:64] FLAG: --pod-cidr="" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758335 2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758341 2550 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758344 2550 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758347 2550 flags.go:64] FLAG: --pods-per-core="0" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758350 2550 flags.go:64] FLAG: --port="10250" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758353 2550 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758356 2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a503cb14e8b90cde" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758359 2550 flags.go:64] FLAG: --qos-reserved="" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758362 2550 flags.go:64] FLAG: --read-only-port="10255" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758364 2550 flags.go:64] FLAG: --register-node="true" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758367 2550 flags.go:64] FLAG: --register-with-taints="" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758371 2550 flags.go:64] FLAG: --registry-burst="10" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758374 2550 flags.go:64] FLAG: --registry-qps="5" Sep 09 21:09:57.764372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758377 2550 flags.go:64] FLAG: --reserved-cpus="" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758380 2550 flags.go:64] FLAG: --reserved-memory="" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758384 2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758387 2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758390 2550 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758392 2550 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758395 2550 flags.go:64] FLAG: --runonce="false" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758401 2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758403 2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758407 2550 flags.go:64] FLAG: --seccomp-default="false" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758409 2550 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758412 2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758415 2550 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758418 2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758421 2550 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758424 2550 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758427 2550 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758430 2550 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758433 2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758436 2550 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758439 2550 flags.go:64] FLAG: --system-cgroups="" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758441 2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758447 2550 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758449 2550 flags.go:64] FLAG: --tls-cert-file="" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758452 2550 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 21:09:57.764961 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758457 2550 flags.go:64] FLAG: --tls-min-version="" Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758459 2550 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758462 2550 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758465 2550 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758468 2550 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758471 2550 flags.go:64] FLAG: --v="2" Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758476 2550 flags.go:64] FLAG: --version="false" Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758480 2550 flags.go:64] FLAG: --vmodule="" Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758484 2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.758488 2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758626 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758631 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758635 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758639 2550 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758643 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758647 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758650 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758653 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758656 2550 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758659 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758662 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758664 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:57.765571 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758667 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758670 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758672 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758675 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758678 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758681 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758683 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758687 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758690 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758692 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758695 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758697 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758700 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758702 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758705 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758709 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758713 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758716 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758719 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758722 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:57.766099 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758725 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758728 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758731 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758735 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758739 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758742 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758745 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758748 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758750 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758753 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758755 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758758 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758760 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758762 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758765 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758768 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758770 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758773 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758775 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758791 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:57.766596 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758794 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758807 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758810 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758812 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758815 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758818 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758821 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758823 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758826 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758828 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758831 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758834 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758837 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758840 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758842 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758845 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758849 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758852 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758855 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758857 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:57.767092 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758860 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758862 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758864 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758867 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758869 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758872 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758874 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758877 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758880 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758882 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758885 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758889 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758892 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758894 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758897 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758899 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758902 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758904 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758907 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:57.767605 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758909 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:57.768111 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758912 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:57.768111 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758914 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:57.768111 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758916 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:57.768111 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758919 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:57.768111 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758922 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:57.768111 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758925 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:57.768111 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.758927 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:57.768111 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.759655 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:09:57.771675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.771533 2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 21:09:57.771675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.771674 2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 21:09:57.771754 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.771690 2550 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 21:09:57.771754 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.771696 2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771755 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771762 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771765 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771768 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771771 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771774 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771777 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771779 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771782 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771784 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771787 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771789 2550 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771792 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771794 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771797 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771800 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771802 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771804 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:57.771805 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771807 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771810 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771813 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771815 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771819 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771821 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771824 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771826 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771829 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771831 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771834 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771837 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771840 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771843 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771845 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771848 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771851 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771853 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771856 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771858 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:57.772276 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771860 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771863 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771865 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771868 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771870 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771872 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771875 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771877 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771880 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771882 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771885 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771887 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771889 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771893 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771898 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771901 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771905 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771908 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771911 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:57.772787 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771914 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771917 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771920 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771923 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771925 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771929 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771931 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771934 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771937 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771939 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771941 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771944 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771948 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771951 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771954 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771958 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771960 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771963 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771966 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:57.773237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771969 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771971 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771973 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771976 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771979 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771982 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771984 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771986 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771990 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771992 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771995 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.771998 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772000 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772003 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772005 2550 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772008 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772011 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772013 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772015 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772019 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:57.773703 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772021 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772023 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772026 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.772031 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772143 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772148 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772151 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772155 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772159 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772162 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772165 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772168 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772171 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772173 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:57.774184 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772176 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772179 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772190 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772194 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772197 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772200 2550 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772203 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772206 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772209 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772212 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772214 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772217 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772220 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772222 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772225 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772227 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772230 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772232 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772236 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772238 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:57.774654 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772241 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772243 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772246 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772248 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772251 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772253 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772256 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772258 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772260 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772263 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772266 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772268 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772271 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772273 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772275 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772278 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772281 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772284 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772289 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772293 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:57.775169 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772295 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772298 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772301 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772304 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772306 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772309 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772311 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772314 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772330 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772333 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772336 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772339 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772342 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772345 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772349 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772351 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772354 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772356 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772359 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:57.775714 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772361 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772364 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772366 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772369 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772371 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772374 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772376 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772379 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772381 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772384 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772387 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772390 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772393 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772396 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772399 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772402 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772405 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772407 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772410 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772413 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:57.776180 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772415 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772418 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772421 2550 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772423 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772426 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772429 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772432 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772434 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772437 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:57.772439 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.772445 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.772566 2550 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 21:09:57.776697 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.775423 2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 21:09:57.776990 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.776872 2550 server.go:1019] "Starting client certificate rotation" Sep 09 21:09:57.776990 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.776968 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 21:09:57.777045 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.777003 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 21:09:57.779061 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.779048 2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 21:09:57.780200 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.780176 2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 21:09:57.789285 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.789263 2550 log.go:25] "Validated CRI v1 runtime API" Sep 09 21:09:57.794154 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.794136 2550 log.go:25] "Validated CRI v1 image API" Sep 09 21:09:57.795084 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.795068 2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 21:09:57.798692 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.798669 2550 fs.go:135] Filesystem UUIDs: map[0b9f96a7-8936-486d-b7c6-92144a1b2b59:/dev/nvme0n1p3 304e7c9b-b9c9-4587-8ca1-0556cf64daac:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Sep 09 21:09:57.798778 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.798691 2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 21:09:57.805189 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.805069 2550 manager.go:216] Machine: {Timestamp:2026-09-09 21:09:57.802873817 +0000 UTC m=+0.315610638 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095032 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d6fb9ff913f95d05676ea70f059ce SystemUUID:ec2d6fb9-ff91-3f95-d056-76ea70f059ce BootID:66ad8699-1731-4f41-b782-8a39a5e71477 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:94:d8:22:13:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:94:d8:22:13:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:e9:38:1c:e7:10 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 21:09:57.805189 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.805183 2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 21:09:57.805350 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.805337 2550 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 21:09:57.809861 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.809826 2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 21:09:57.810005 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.809865 2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-60.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 21:09:57.810584 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.810573 2550 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 21:09:57.810620 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.810586 2550 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 21:09:57.810620 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.810594 2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 21:09:57.811967 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.811952 2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 21:09:57.813245 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.813233 2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 21:09:57.814074 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.814058 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 21:09:57.817525 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.817509 2550 state_mem.go:36] "Initialized new in-memory state store" Sep 09 21:09:57.817659 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.817649 2550 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 21:09:57.817693 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.817680 2550 kubelet.go:486] "Attempting to sync node with API server" Sep 09 21:09:57.817726 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.817693 2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 21:09:57.817726 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.817710 2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 21:09:57.817726 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.817726 2550 kubelet.go:398] "Adding apiserver pod source" Sep 09 21:09:57.817822 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.817738 2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 21:09:57.818107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.818095 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 21:09:57.818146 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.818112 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 21:09:57.820400 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.820376 2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 21:09:57.822290 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.822273 2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 21:09:57.822390 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.822312 2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 21:09:57.823536 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823514 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 21:09:57.823536 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823543 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 21:09:57.823667 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823552 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 21:09:57.823667 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823560 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 21:09:57.823667 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823572 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 21:09:57.823667 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823580 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 21:09:57.823667 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823599 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 21:09:57.823667 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823608 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 21:09:57.823667 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823620 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 21:09:57.823667 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823629 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 21:09:57.823667 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823641 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 21:09:57.823667 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.823651 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 21:09:57.825000 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.824988 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 21:09:57.825052 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.825004 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 21:09:57.828946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.828929 2550 server.go:1268] "Started kubelet" Sep 09 21:09:57.829060 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.829014 2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 21:09:57.829109 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.829047 2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 21:09:57.829109 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.829087 2550 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 21:09:57.829728 ip-10-0-128-60 systemd[1]: Started Kubernetes Kubelet. Sep 09 21:09:57.833188 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.833164 2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 21:09:57.834438 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.834418 2550 server.go:310] "Adding debug handlers to kubelet server" Sep 09 21:09:57.834546 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.834521 2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 21:09:57.837968 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.837935 2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 21:09:57.837968 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.837942 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 21:09:57.838532 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.838249 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:57.838532 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.838439 2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 21:09:57.838532 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.838456 2550 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 21:09:57.838532 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.838484 2550 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 21:09:57.838713 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.838545 2550 reconstruct.go:97] "Volume reconstruction finished" Sep 09 21:09:57.838713 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.838557 2550 reconciler.go:29] "Reconciler: start to sync state" Sep 09 21:09:57.838857 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.838835 2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 21:09:57.838857 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.838857 2550 factory.go:55] Registering systemd factory Sep 09 21:09:57.839020 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.838867 2550 factory.go:223] Registration of the systemd container factory successfully Sep 09 21:09:57.839108 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.839095 2550 factory.go:153] Registering CRI-O factory Sep 09 21:09:57.839158 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.839111 2550 factory.go:223] Registration of the crio container factory successfully Sep 09 21:09:57.839158 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.839127 2550 factory.go:103] Registering Raw factory Sep 09 21:09:57.839158 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.839140 2550 manager.go:1196] Started watching for new ooms in manager Sep 09 21:09:57.839842 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.839550 2550 manager.go:349] Starting recovery of all containers Sep 09 21:09:57.845952 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.845917 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 21:09:57.846094 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.846067 2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-60.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 21:09:57.846183 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.846160 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-60.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 21:09:57.846253 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.846191 2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-60.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 21:09:57.846344 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.846302 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 21:09:57.846686 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.845872 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-60.ec2.internal.18d3c34996f53bed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-60.ec2.internal,UID:ip-10-0-128-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-60.ec2.internal,},FirstTimestamp:2026-09-09 21:09:57.828901869 +0000 UTC m=+0.341638701,LastTimestamp:2026-09-09 21:09:57.828901869 +0000 UTC m=+0.341638701,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-60.ec2.internal,}" Sep 09 21:09:57.848866 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.848821 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 21:09:57.849578 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.849560 2550 manager.go:354] Recovery completed Sep 09 21:09:57.850873 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.850792 2550 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 09 21:09:57.854017 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.854001 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:57.855868 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.855844 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:57.855924 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.855881 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:57.855924 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.855892 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:57.856784 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.856765 2550 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 21:09:57.856871 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.856787 2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 21:09:57.856871 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.856806 2550 state_mem.go:36] "Initialized new in-memory state store" Sep 09 21:09:57.856975 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.856966 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:57.857910 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.857871 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:57.857910 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.857908 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:57.858029 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.857922 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:57.858871 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.858852 2550 policy_none.go:49] "None policy: Start" Sep 09 21:09:57.858942 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.858875 2550 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 21:09:57.858942 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.858890 2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 21:09:57.859274 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.859200 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-60.ec2.internal.18d3c3499890b32d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-60.ec2.internal,UID:ip-10-0-128-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-60.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-60.ec2.internal,},FirstTimestamp:2026-09-09 21:09:57.855867693 +0000 UTC m=+0.368604502,LastTimestamp:2026-09-09 21:09:57.855867693 +0000 UTC m=+0.368604502,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-60.ec2.internal,}" Sep 09 21:09:57.860645 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.860631 2550 policy_none.go:47] "Start" Sep 09 21:09:57.870049 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.869976 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-60.ec2.internal.18d3c3499890f9be default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-60.ec2.internal,UID:ip-10-0-128-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-60.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-60.ec2.internal,},FirstTimestamp:2026-09-09 21:09:57.855885758 +0000 UTC m=+0.368622568,LastTimestamp:2026-09-09 21:09:57.855885758 +0000 UTC m=+0.368622568,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-60.ec2.internal,}" Sep 09 21:09:57.873835 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.873818 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qkm9h" Sep 09 21:09:57.879608 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.879536 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-60.ec2.internal.18d3c3499891213a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-60.ec2.internal,UID:ip-10-0-128-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-60.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-60.ec2.internal,},FirstTimestamp:2026-09-09 21:09:57.855895866 +0000 UTC m=+0.368632676,LastTimestamp:2026-09-09 21:09:57.855895866 +0000 UTC m=+0.368632676,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-60.ec2.internal,}" Sep 09 21:09:57.882847 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.882830 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qkm9h" Sep 09 21:09:57.884752 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.884678 2550 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-60.ec2.internal.18d3c3499890b32d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-60.ec2.internal.18d3c3499890b32d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-60.ec2.internal,UID:ip-10-0-128-60.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-60.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-60.ec2.internal,},FirstTimestamp:2026-09-09 21:09:57.855867693 +0000 UTC m=+0.368604502,LastTimestamp:2026-09-09 21:09:57.857888155 +0000 UTC m=+0.370624965,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-60.ec2.internal,}" Sep 09 21:09:57.903105 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.903088 2550 manager.go:337] "Starting Device Plugin manager" Sep 09 21:09:57.928351 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.903130 2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 21:09:57.928351 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.903141 2550 server.go:87] "Starting device plugin registration server" Sep 09 21:09:57.928351 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.903446 2550 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 21:09:57.928351 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.903459 2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 21:09:57.928351 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.903555 2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 21:09:57.928351 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.903645 2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 21:09:57.928351 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.903653 2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 21:09:57.928351 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.904224 2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 21:09:57.928351 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.904282 2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:57.984064 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.984026 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 21:09:57.984237 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.984092 2550 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 21:09:57.984237 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.984115 2550 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 21:09:57.984237 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:57.984157 2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 21:09:57.986159 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:57.986141 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:09:58.004443 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.004343 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:58.005277 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.005261 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:58.005373 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.005293 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:58.005373 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.005306 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:58.005373 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.005348 2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.014454 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.014426 2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.014454 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.014453 2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-60.ec2.internal\": node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:58.035223 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.035202 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:58.085034 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.084989 2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-60.ec2.internal"] Sep 09 21:09:58.085126 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.085096 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:58.086112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.086085 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:58.086237 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.086121 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:58.086237 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.086137 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:58.087485 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.087470 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:58.087594 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.087579 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.087641 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.087620 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:58.088222 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.088208 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:58.088296 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.088241 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:58.088296 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.088245 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:58.088296 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.088256 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:58.088296 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.088270 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:58.088296 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.088280 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:58.089576 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.089563 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.089628 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.089587 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:58.090237 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.090221 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:58.090303 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.090243 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:58.090303 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.090253 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:58.111082 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.111058 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-60.ec2.internal\" not found" node="ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.115246 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.115228 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-60.ec2.internal\" not found" node="ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.135578 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.135544 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:58.139873 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.139853 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee47118de8909838405186e61d51474b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal\" (UID: \"ee47118de8909838405186e61d51474b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.139954 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.139880 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee47118de8909838405186e61d51474b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal\" (UID: \"ee47118de8909838405186e61d51474b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.139954 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.139899 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d53d5013d65214a919bdfb5b967b3585-config\") pod \"kube-apiserver-proxy-ip-10-0-128-60.ec2.internal\" (UID: \"d53d5013d65214a919bdfb5b967b3585\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.235842 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.235802 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:58.240102 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.240082 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee47118de8909838405186e61d51474b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal\" (UID: \"ee47118de8909838405186e61d51474b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.240198 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.240123 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee47118de8909838405186e61d51474b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal\" (UID: \"ee47118de8909838405186e61d51474b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.240198 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.240133 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d53d5013d65214a919bdfb5b967b3585-config\") pod \"kube-apiserver-proxy-ip-10-0-128-60.ec2.internal\" (UID: \"d53d5013d65214a919bdfb5b967b3585\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.240198 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.240161 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee47118de8909838405186e61d51474b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal\" (UID: \"ee47118de8909838405186e61d51474b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.240198 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.240189 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d53d5013d65214a919bdfb5b967b3585-config\") pod \"kube-apiserver-proxy-ip-10-0-128-60.ec2.internal\" (UID: \"d53d5013d65214a919bdfb5b967b3585\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.240368 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.240216 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee47118de8909838405186e61d51474b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal\" (UID: \"ee47118de8909838405186e61d51474b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.336613 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.336546 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:58.414048 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.414018 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.418729 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.418712 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-60.ec2.internal" Sep 09 21:09:58.437420 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.437387 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:58.537909 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.537873 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:58.638379 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.638293 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:58.738764 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.738731 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:58.776105 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.776077 2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 21:09:58.776525 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.776236 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 21:09:58.839703 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.839672 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:58.878537 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.878513 2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:09:58.885709 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.885670 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 21:04:58 +0000 UTC" deadline="2028-04-09 03:21:39.113157498 +0000 UTC" Sep 09 21:09:58.885709 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.885705 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13854h11m40.22745594s" Sep 09 21:09:58.936038 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.935962 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:58.938893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.938871 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:58.939010 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.938905 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:58.939010 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.938915 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:58.939956 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:58.939940 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:58.949425 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:58.949402 2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:09:59.027210 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:59.026975 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd53d5013d65214a919bdfb5b967b3585.slice/crio-c0b434420a06dd3dc50326b9374ab42f0086cfd1a8b1fc8a62cbe1493bd695bb WatchSource:0}: Error finding container c0b434420a06dd3dc50326b9374ab42f0086cfd1a8b1fc8a62cbe1493bd695bb: Status 404 returned error can't find the container with id c0b434420a06dd3dc50326b9374ab42f0086cfd1a8b1fc8a62cbe1493bd695bb Sep 09 21:09:59.027472 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:09:59.027448 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee47118de8909838405186e61d51474b.slice/crio-e669fc0486c8676e9169f86536e4a684177f194bacc503a6f186b00115969505 WatchSource:0}: Error finding container e669fc0486c8676e9169f86536e4a684177f194bacc503a6f186b00115969505: Status 404 returned error can't find the container with id e669fc0486c8676e9169f86536e4a684177f194bacc503a6f186b00115969505 Sep 09 21:09:59.031874 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.031858 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:09:59.040785 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:59.040759 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:59.141834 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:59.141793 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:59.242181 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:59.242094 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-60.ec2.internal\" not found" Sep 09 21:09:59.249974 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.249952 2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:09:59.338879 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.338848 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-60.ec2.internal" Sep 09 21:09:59.350589 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.350563 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 21:09:59.351649 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.351635 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" Sep 09 21:09:59.363684 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.363661 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 21:09:59.818814 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.818780 2550 apiserver.go:52] "Watching apiserver" Sep 09 21:09:59.826259 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.826230 2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 21:09:59.828778 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.828743 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-h8h2f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal","openshift-multus/multus-9lzjh","openshift-multus/multus-additional-cni-plugins-vth7c","openshift-multus/network-metrics-daemon-9jn8c","openshift-network-operator/iptables-alerter-dfjgf","openshift-ovn-kubernetes/ovnkube-node-fr2g6","kube-system/global-pull-secret-syncer-kggbk","kube-system/konnectivity-agent-qrjzg","kube-system/kube-apiserver-proxy-ip-10-0-128-60.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr","openshift-cluster-node-tuning-operator/tuned-qk7vk","openshift-image-registry/node-ca-rx56w","openshift-network-diagnostics/network-check-target-xctpc"] Sep 09 21:09:59.831763 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.831736 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.834426 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.834388 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rx56w" Sep 09 21:09:59.835256 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.835222 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 21:09:59.835256 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.835240 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 21:09:59.835426 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.835281 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 21:09:59.835426 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.835227 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 21:09:59.835426 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.835364 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 21:09:59.835767 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.835747 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r2mr6\"" Sep 09 21:09:59.835847 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.835833 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 21:09:59.836528 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.836511 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 21:09:59.836889 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.836870 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-frb82\"" Sep 09 21:09:59.836973 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.836951 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 21:09:59.837175 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.837157 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 21:09:59.838166 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.837726 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.838706 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.838677 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 21:09:59.839565 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.839538 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-57m4h\"" Sep 09 21:09:59.839897 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.839879 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 21:09:59.840096 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.840080 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 21:09:59.840282 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.840267 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 21:09:59.840970 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.840783 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 21:09:59.840970 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.840957 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.842841 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.842817 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 21:09:59.843108 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.843088 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 21:09:59.843427 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.843284 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4fshk\"" Sep 09 21:09:59.844327 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.844129 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:09:59.844327 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:59.844203 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:09:59.845227 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.845206 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dfjgf" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.847450 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.848581 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fl4ll\"" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.848799 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.849655 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.849775 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850061 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a04424e-a63f-45bc-bd8b-53b3281e0982-cni-binary-copy\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850096 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-etc-openvswitch\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850127 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-run-ovn\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850173 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-node-log\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850203 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-log-socket\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850229 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2cpr\" (UniqueName: \"kubernetes.io/projected/71354de2-a7c4-4336-9a2f-74cf7755fadb-kube-api-access-c2cpr\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850277 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-systemd-units\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850339 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71354de2-a7c4-4336-9a2f-74cf7755fadb-env-overrides\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850384 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-cnibin\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850432 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-socket-dir-parent\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850478 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-run-netns\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850505 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-hostroot\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850546 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-daemon-config\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.850675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850578 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a04424e-a63f-45bc-bd8b-53b3281e0982-system-cni-dir\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850654 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a04424e-a63f-45bc-bd8b-53b3281e0982-cnibin\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850944 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a04424e-a63f-45bc-bd8b-53b3281e0982-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.850977 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fdd3337d-7102-4a4b-826f-093466b1014d-serviceca\") pod \"node-ca-rx56w\" (UID: \"fdd3337d-7102-4a4b-826f-093466b1014d\") " pod="openshift-image-registry/node-ca-rx56w" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851013 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-system-cni-dir\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851122 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851196 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-cni-dir\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851217 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a329dfed-fd34-425d-8372-fd0db6ac6bd7-cni-binary-copy\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851236 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x6zj\" (UniqueName: \"kubernetes.io/projected/a329dfed-fd34-425d-8372-fd0db6ac6bd7-kube-api-access-6x6zj\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851256 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-slash\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851286 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h8h2f" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851292 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-run-netns\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851312 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-run-ovn-kubernetes\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.851422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851358 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71354de2-a7c4-4336-9a2f-74cf7755fadb-ovn-node-metrics-cert\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851419 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a04424e-a63f-45bc-bd8b-53b3281e0982-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851463 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-cni-bin\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851488 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fdd3337d-7102-4a4b-826f-093466b1014d-host\") pod \"node-ca-rx56w\" (UID: \"fdd3337d-7102-4a4b-826f-093466b1014d\") " pod="openshift-image-registry/node-ca-rx56w" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851518 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr5sk\" (UniqueName: \"kubernetes.io/projected/fdd3337d-7102-4a4b-826f-093466b1014d-kube-api-access-zr5sk\") pod \"node-ca-rx56w\" (UID: \"fdd3337d-7102-4a4b-826f-093466b1014d\") " pod="openshift-image-registry/node-ca-rx56w" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851552 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-run-k8s-cni-cncf-io\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851584 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-run-multus-certs\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851609 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-etc-kubernetes\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851692 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a04424e-a63f-45bc-bd8b-53b3281e0982-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851756 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-kubelet\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851801 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-var-lib-openvswitch\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851849 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71354de2-a7c4-4336-9a2f-74cf7755fadb-ovnkube-config\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851889 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-conf-dir\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851925 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-run-systemd\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.852004 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.851992 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-var-lib-kubelet\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.852657 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.852113 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nvwh\" (UniqueName: \"kubernetes.io/projected/0a04424e-a63f-45bc-bd8b-53b3281e0982-kube-api-access-5nvwh\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.852657 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.852158 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-run-openvswitch\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.852657 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.852221 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.852657 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.852260 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71354de2-a7c4-4336-9a2f-74cf7755fadb-ovnkube-script-lib\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.852657 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.852309 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-var-lib-cni-bin\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.852657 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.852402 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-var-lib-cni-multus\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.852657 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.852434 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-cni-netd\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.852657 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.852625 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xh4r9\"" Sep 09 21:09:59.853061 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.852626 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 21:09:59.853526 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.853505 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 21:09:59.854055 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.854014 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 21:09:59.854144 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.854113 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wczkl\"" Sep 09 21:09:59.854475 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.854018 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:09:59.854475 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:59.854457 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:09:59.854776 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.854742 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 21:09:59.858626 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.858603 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:09:59.858727 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:09:59.858675 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:09:59.861421 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.861396 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:09:59.864052 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.864032 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.865220 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.865200 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 21:09:59.865977 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.865960 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 21:09:59.866497 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.866477 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2l7cg\"" Sep 09 21:09:59.866675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.866655 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 21:09:59.871371 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.871331 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 21:09:59.871371 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.871344 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:09:59.873428 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.873408 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9gvrx\"" Sep 09 21:09:59.884202 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.884180 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gqkcl" Sep 09 21:09:59.893434 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.893018 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gqkcl" Sep 09 21:09:59.939599 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.939562 2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 21:09:59.952861 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.952733 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:09:59.952861 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.952782 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-kubernetes\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.952861 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.952843 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8885188-4e52-4325-8d27-8f3ab4700578-konnectivity-ca\") pod \"konnectivity-agent-qrjzg\" (UID: \"e8885188-4e52-4325-8d27-8f3ab4700578\") " pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:09:59.953161 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.952890 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4211bff4-82e4-4201-b854-de56fd0b7e14-dbus\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:09:59.953161 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.952926 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71354de2-a7c4-4336-9a2f-74cf7755fadb-ovnkube-config\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.953161 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.952988 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3377582-07d8-484f-b42e-01b20a11dc48-etc-tuned\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.953161 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953036 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9018849b-d1a1-4fcc-8854-4641e9a87cfe-iptables-alerter-script\") pod \"iptables-alerter-dfjgf\" (UID: \"9018849b-d1a1-4fcc-8854-4641e9a87cfe\") " pod="openshift-network-operator/iptables-alerter-dfjgf" Sep 09 21:09:59.953161 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953085 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69lvv\" (UniqueName: \"kubernetes.io/projected/4b343c4d-2b12-4093-b6de-c4223e67f2bf-kube-api-access-69lvv\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:09:59.953161 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953140 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-run-systemd\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.953632 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953197 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-run-systemd\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.953632 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953229 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-sysctl-conf\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.953632 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953273 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-run\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.953632 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953304 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9018849b-d1a1-4fcc-8854-4641e9a87cfe-host-slash\") pod \"iptables-alerter-dfjgf\" (UID: \"9018849b-d1a1-4fcc-8854-4641e9a87cfe\") " pod="openshift-network-operator/iptables-alerter-dfjgf" Sep 09 21:09:59.953632 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953334 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ab41251-7b22-458e-83c1-4dc1ed95611a-hosts-file\") pod \"node-resolver-h8h2f\" (UID: \"4ab41251-7b22-458e-83c1-4dc1ed95611a\") " pod="openshift-dns/node-resolver-h8h2f" Sep 09 21:09:59.953632 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953424 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-var-lib-cni-multus\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.953632 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953456 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-log-socket\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.953632 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953480 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2cpr\" (UniqueName: \"kubernetes.io/projected/71354de2-a7c4-4336-9a2f-74cf7755fadb-kube-api-access-c2cpr\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.953632 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953553 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-log-socket\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.953632 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953585 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmlvv\" (UniqueName: \"kubernetes.io/projected/9018849b-d1a1-4fcc-8854-4641e9a87cfe-kube-api-access-xmlvv\") pod \"iptables-alerter-dfjgf\" (UID: \"9018849b-d1a1-4fcc-8854-4641e9a87cfe\") " pod="openshift-network-operator/iptables-alerter-dfjgf" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953632 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-hostroot\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953676 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3377582-07d8-484f-b42e-01b20a11dc48-tmp\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953726 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-hostroot\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953758 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953789 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a04424e-a63f-45bc-bd8b-53b3281e0982-system-cni-dir\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953817 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a04424e-a63f-45bc-bd8b-53b3281e0982-cnibin\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953845 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a04424e-a63f-45bc-bd8b-53b3281e0982-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953875 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a329dfed-fd34-425d-8372-fd0db6ac6bd7-cni-binary-copy\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.953901 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-sysconfig\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954013 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71354de2-a7c4-4336-9a2f-74cf7755fadb-ovnkube-config\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954087 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-var-lib-cni-multus\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.954112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954112 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-slash\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.954562 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954129 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-run-ovn-kubernetes\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.954562 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954147 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71354de2-a7c4-4336-9a2f-74cf7755fadb-ovn-node-metrics-cert\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.954562 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954164 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-lib-modules\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.954562 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954184 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqrgx\" (UniqueName: \"kubernetes.io/projected/4ab41251-7b22-458e-83c1-4dc1ed95611a-kube-api-access-nqrgx\") pod \"node-resolver-h8h2f\" (UID: \"4ab41251-7b22-458e-83c1-4dc1ed95611a\") " pod="openshift-dns/node-resolver-h8h2f" Sep 09 21:09:59.954562 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954229 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-sys-fs\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:09:59.954562 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954257 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a04424e-a63f-45bc-bd8b-53b3281e0982-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.954562 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954291 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-run-k8s-cni-cncf-io\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.954562 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954314 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-run-multus-certs\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.954831 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954590 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a329dfed-fd34-425d-8372-fd0db6ac6bd7-cni-binary-copy\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.954831 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954694 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-run-ovn-kubernetes\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.954831 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.954749 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-slash\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.955267 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955247 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a04424e-a63f-45bc-bd8b-53b3281e0982-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.955358 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955327 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-run-k8s-cni-cncf-io\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.955426 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955412 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-run-multus-certs\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.955479 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955450 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a04424e-a63f-45bc-bd8b-53b3281e0982-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.955479 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955448 2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 21:09:59.955592 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955482 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-kubelet\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.955592 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955509 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-var-lib-openvswitch\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.955697 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955616 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-conf-dir\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.955697 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955646 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-var-lib-kubelet\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.955697 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955675 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-registration-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:09:59.955822 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955712 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-device-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:09:59.955822 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955746 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:09:59.955822 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955771 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-sysctl-d\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.955915 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955824 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-var-lib-openvswitch\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.955915 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955832 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ab41251-7b22-458e-83c1-4dc1ed95611a-tmp-dir\") pod \"node-resolver-h8h2f\" (UID: \"4ab41251-7b22-458e-83c1-4dc1ed95611a\") " pod="openshift-dns/node-resolver-h8h2f" Sep 09 21:09:59.955977 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955915 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-kubelet\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.955977 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955966 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a04424e-a63f-45bc-bd8b-53b3281e0982-cnibin\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.956040 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.955984 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-conf-dir\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.956074 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956059 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-var-lib-kubelet\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.956137 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956117 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb97w\" (UniqueName: \"kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w\") pod \"network-check-target-xctpc\" (UID: \"537bb9e7-2c10-4fcf-a29c-352599686ffa\") " pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:09:59.956183 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956167 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nvwh\" (UniqueName: \"kubernetes.io/projected/0a04424e-a63f-45bc-bd8b-53b3281e0982-kube-api-access-5nvwh\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.956233 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956166 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a04424e-a63f-45bc-bd8b-53b3281e0982-system-cni-dir\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.956233 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956123 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a04424e-a63f-45bc-bd8b-53b3281e0982-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.956417 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956378 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-run-openvswitch\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.956417 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956379 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a04424e-a63f-45bc-bd8b-53b3281e0982-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.956533 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956519 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-run-openvswitch\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.956588 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956547 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.956638 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956611 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.957107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956731 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71354de2-a7c4-4336-9a2f-74cf7755fadb-ovnkube-script-lib\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.957107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956800 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-var-lib-cni-bin\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.957107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956849 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-modprobe-d\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.957107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956864 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-var-lib-cni-bin\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.957107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956870 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-var-lib-kubelet\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.957107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956928 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56hn5\" (UniqueName: \"kubernetes.io/projected/b3377582-07d8-484f-b42e-01b20a11dc48-kube-api-access-56hn5\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.957107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.956963 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-cni-netd\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.957107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957012 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a04424e-a63f-45bc-bd8b-53b3281e0982-cni-binary-copy\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.957107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957050 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-etc-openvswitch\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.957107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957049 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-cni-netd\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.957107 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957095 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-etc-openvswitch\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.957685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957345 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71354de2-a7c4-4336-9a2f-74cf7755fadb-ovnkube-script-lib\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.957685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957420 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-run-ovn\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.957685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957451 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-node-log\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.957685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957485 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-cnibin\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.957685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957509 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-socket-dir-parent\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.957685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957535 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-socket-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:09:59.957685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957571 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9nst\" (UniqueName: \"kubernetes.io/projected/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-kube-api-access-n9nst\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:09:59.957685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957578 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a04424e-a63f-45bc-bd8b-53b3281e0982-cni-binary-copy\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.957685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957597 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-systemd-units\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.957685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957634 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71354de2-a7c4-4336-9a2f-74cf7755fadb-env-overrides\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.957685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957660 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-run-netns\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957725 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-run-ovn\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957774 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-socket-dir-parent\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957773 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-daemon-config\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957807 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-cnibin\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957817 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x6zj\" (UniqueName: \"kubernetes.io/projected/a329dfed-fd34-425d-8372-fd0db6ac6bd7-kube-api-access-6x6zj\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957852 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-systemd-units\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957873 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8885188-4e52-4325-8d27-8f3ab4700578-agent-certs\") pod \"konnectivity-agent-qrjzg\" (UID: \"e8885188-4e52-4325-8d27-8f3ab4700578\") " pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957912 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-node-log\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957963 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fdd3337d-7102-4a4b-826f-093466b1014d-serviceca\") pod \"node-ca-rx56w\" (UID: \"fdd3337d-7102-4a4b-826f-093466b1014d\") " pod="openshift-image-registry/node-ca-rx56w" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.957985 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-host-run-netns\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958015 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-system-cni-dir\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958055 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-cni-dir\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958087 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-systemd\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958122 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958161 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-run-netns\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958233 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-sys\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958262 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-host\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:09:59.958395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958271 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-daemon-config\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958376 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4211bff4-82e4-4201-b854-de56fd0b7e14-kubelet-config\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958477 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71354de2-a7c4-4336-9a2f-74cf7755fadb-env-overrides\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958514 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-system-cni-dir\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958501 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-multus-cni-dir\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958537 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-run-netns\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958548 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-cni-bin\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958595 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fdd3337d-7102-4a4b-826f-093466b1014d-host\") pod \"node-ca-rx56w\" (UID: \"fdd3337d-7102-4a4b-826f-093466b1014d\") " pod="openshift-image-registry/node-ca-rx56w" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958642 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71354de2-a7c4-4336-9a2f-74cf7755fadb-host-cni-bin\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958650 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr5sk\" (UniqueName: \"kubernetes.io/projected/fdd3337d-7102-4a4b-826f-093466b1014d-kube-api-access-zr5sk\") pod \"node-ca-rx56w\" (UID: \"fdd3337d-7102-4a4b-826f-093466b1014d\") " pod="openshift-image-registry/node-ca-rx56w" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958679 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fdd3337d-7102-4a4b-826f-093466b1014d-host\") pod \"node-ca-rx56w\" (UID: \"fdd3337d-7102-4a4b-826f-093466b1014d\") " pod="openshift-image-registry/node-ca-rx56w" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958719 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-etc-kubernetes\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958806 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fdd3337d-7102-4a4b-826f-093466b1014d-serviceca\") pod \"node-ca-rx56w\" (UID: \"fdd3337d-7102-4a4b-826f-093466b1014d\") " pod="openshift-image-registry/node-ca-rx56w" Sep 09 21:09:59.959389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.958878 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a329dfed-fd34-425d-8372-fd0db6ac6bd7-etc-kubernetes\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.963446 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.963426 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71354de2-a7c4-4336-9a2f-74cf7755fadb-ovn-node-metrics-cert\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.965403 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.965298 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2cpr\" (UniqueName: \"kubernetes.io/projected/71354de2-a7c4-4336-9a2f-74cf7755fadb-kube-api-access-c2cpr\") pod \"ovnkube-node-fr2g6\" (UID: \"71354de2-a7c4-4336-9a2f-74cf7755fadb\") " pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:09:59.967019 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.966957 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nvwh\" (UniqueName: \"kubernetes.io/projected/0a04424e-a63f-45bc-bd8b-53b3281e0982-kube-api-access-5nvwh\") pod \"multus-additional-cni-plugins-vth7c\" (UID: \"0a04424e-a63f-45bc-bd8b-53b3281e0982\") " pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:09:59.968220 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.967802 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x6zj\" (UniqueName: \"kubernetes.io/projected/a329dfed-fd34-425d-8372-fd0db6ac6bd7-kube-api-access-6x6zj\") pod \"multus-9lzjh\" (UID: \"a329dfed-fd34-425d-8372-fd0db6ac6bd7\") " pod="openshift-multus/multus-9lzjh" Sep 09 21:09:59.970284 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.970254 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr5sk\" (UniqueName: \"kubernetes.io/projected/fdd3337d-7102-4a4b-826f-093466b1014d-kube-api-access-zr5sk\") pod \"node-ca-rx56w\" (UID: \"fdd3337d-7102-4a4b-826f-093466b1014d\") " pod="openshift-image-registry/node-ca-rx56w" Sep 09 21:09:59.989931 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.989858 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" event={"ID":"ee47118de8909838405186e61d51474b","Type":"ContainerStarted","Data":"e669fc0486c8676e9169f86536e4a684177f194bacc503a6f186b00115969505"} Sep 09 21:09:59.991786 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:09:59.991756 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-60.ec2.internal" event={"ID":"d53d5013d65214a919bdfb5b967b3585","Type":"ContainerStarted","Data":"c0b434420a06dd3dc50326b9374ab42f0086cfd1a8b1fc8a62cbe1493bd695bb"} Sep 09 21:10:00.059136 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059096 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-run\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.059136 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059143 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9018849b-d1a1-4fcc-8854-4641e9a87cfe-host-slash\") pod \"iptables-alerter-dfjgf\" (UID: \"9018849b-d1a1-4fcc-8854-4641e9a87cfe\") " pod="openshift-network-operator/iptables-alerter-dfjgf" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059163 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ab41251-7b22-458e-83c1-4dc1ed95611a-hosts-file\") pod \"node-resolver-h8h2f\" (UID: \"4ab41251-7b22-458e-83c1-4dc1ed95611a\") " pod="openshift-dns/node-resolver-h8h2f" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059192 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmlvv\" (UniqueName: \"kubernetes.io/projected/9018849b-d1a1-4fcc-8854-4641e9a87cfe-kube-api-access-xmlvv\") pod \"iptables-alerter-dfjgf\" (UID: \"9018849b-d1a1-4fcc-8854-4641e9a87cfe\") " pod="openshift-network-operator/iptables-alerter-dfjgf" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059213 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3377582-07d8-484f-b42e-01b20a11dc48-tmp\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059230 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9018849b-d1a1-4fcc-8854-4641e9a87cfe-host-slash\") pod \"iptables-alerter-dfjgf\" (UID: \"9018849b-d1a1-4fcc-8854-4641e9a87cfe\") " pod="openshift-network-operator/iptables-alerter-dfjgf" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059248 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-run\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059237 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059311 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-sysconfig\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059339 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-lib-modules\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059368 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ab41251-7b22-458e-83c1-4dc1ed95611a-hosts-file\") pod \"node-resolver-h8h2f\" (UID: \"4ab41251-7b22-458e-83c1-4dc1ed95611a\") " pod="openshift-dns/node-resolver-h8h2f" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059390 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqrgx\" (UniqueName: \"kubernetes.io/projected/4ab41251-7b22-458e-83c1-4dc1ed95611a-kube-api-access-nqrgx\") pod \"node-resolver-h8h2f\" (UID: \"4ab41251-7b22-458e-83c1-4dc1ed95611a\") " pod="openshift-dns/node-resolver-h8h2f" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.059404 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059420 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-sys-fs\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.059436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059427 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-sysconfig\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059584 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-lib-modules\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.059602 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret podName:4211bff4-82e4-4201-b854-de56fd0b7e14 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:00.559576541 +0000 UTC m=+3.072313353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret") pod "global-pull-secret-syncer-kggbk" (UID: "4211bff4-82e4-4201-b854-de56fd0b7e14") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059694 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-registration-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059735 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-device-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059764 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059786 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-sysctl-d\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059809 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ab41251-7b22-458e-83c1-4dc1ed95611a-tmp-dir\") pod \"node-resolver-h8h2f\" (UID: \"4ab41251-7b22-458e-83c1-4dc1ed95611a\") " pod="openshift-dns/node-resolver-h8h2f" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059832 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb97w\" (UniqueName: \"kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w\") pod \"network-check-target-xctpc\" (UID: \"537bb9e7-2c10-4fcf-a29c-352599686ffa\") " pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059826 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-registration-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059861 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-modprobe-d\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059833 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-sys-fs\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059885 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-var-lib-kubelet\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059925 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56hn5\" (UniqueName: \"kubernetes.io/projected/b3377582-07d8-484f-b42e-01b20a11dc48-kube-api-access-56hn5\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059953 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-socket-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059952 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-var-lib-kubelet\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059971 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9nst\" (UniqueName: \"kubernetes.io/projected/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-kube-api-access-n9nst\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.059986 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-sysctl-d\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060009 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8885188-4e52-4325-8d27-8f3ab4700578-agent-certs\") pod \"konnectivity-agent-qrjzg\" (UID: \"e8885188-4e52-4325-8d27-8f3ab4700578\") " pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060028 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-systemd\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060043 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-device-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060046 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060087 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-sys\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.060095 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060113 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-host\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060138 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4211bff4-82e4-4201-b854-de56fd0b7e14-kubelet-config\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.060154 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs podName:4b343c4d-2b12-4093-b6de-c4223e67f2bf nodeName:}" failed. No retries permitted until 2026-09-09 21:10:00.560141045 +0000 UTC m=+3.072877858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs") pod "network-metrics-daemon-9jn8c" (UID: "4b343c4d-2b12-4093-b6de-c4223e67f2bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060152 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060156 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ab41251-7b22-458e-83c1-4dc1ed95611a-tmp-dir\") pod \"node-resolver-h8h2f\" (UID: \"4ab41251-7b22-458e-83c1-4dc1ed95611a\") " pod="openshift-dns/node-resolver-h8h2f" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060178 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4211bff4-82e4-4201-b854-de56fd0b7e14-kubelet-config\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060204 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-sys\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060229 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-host\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060252 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-systemd\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.060908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060259 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060285 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-kubernetes\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060304 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-socket-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060314 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8885188-4e52-4325-8d27-8f3ab4700578-konnectivity-ca\") pod \"konnectivity-agent-qrjzg\" (UID: \"e8885188-4e52-4325-8d27-8f3ab4700578\") " pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060342 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4211bff4-82e4-4201-b854-de56fd0b7e14-dbus\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060377 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060370 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3377582-07d8-484f-b42e-01b20a11dc48-etc-tuned\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060424 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-modprobe-d\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060442 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9018849b-d1a1-4fcc-8854-4641e9a87cfe-iptables-alerter-script\") pod \"iptables-alerter-dfjgf\" (UID: \"9018849b-d1a1-4fcc-8854-4641e9a87cfe\") " pod="openshift-network-operator/iptables-alerter-dfjgf" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060476 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69lvv\" (UniqueName: \"kubernetes.io/projected/4b343c4d-2b12-4093-b6de-c4223e67f2bf-kube-api-access-69lvv\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060482 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-kubernetes\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060505 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-sysctl-conf\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060668 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b3377582-07d8-484f-b42e-01b20a11dc48-etc-sysctl-conf\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.060918 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4211bff4-82e4-4201-b854-de56fd0b7e14-dbus\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.061010 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8885188-4e52-4325-8d27-8f3ab4700578-konnectivity-ca\") pod \"konnectivity-agent-qrjzg\" (UID: \"e8885188-4e52-4325-8d27-8f3ab4700578\") " pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:10:00.062025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.061044 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9018849b-d1a1-4fcc-8854-4641e9a87cfe-iptables-alerter-script\") pod \"iptables-alerter-dfjgf\" (UID: \"9018849b-d1a1-4fcc-8854-4641e9a87cfe\") " pod="openshift-network-operator/iptables-alerter-dfjgf" Sep 09 21:10:00.062834 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.062324 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3377582-07d8-484f-b42e-01b20a11dc48-tmp\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.063172 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.063148 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8885188-4e52-4325-8d27-8f3ab4700578-agent-certs\") pod \"konnectivity-agent-qrjzg\" (UID: \"e8885188-4e52-4325-8d27-8f3ab4700578\") " pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:10:00.063412 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.063369 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b3377582-07d8-484f-b42e-01b20a11dc48-etc-tuned\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.080945 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.080694 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9nst\" (UniqueName: \"kubernetes.io/projected/8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e-kube-api-access-n9nst\") pod \"aws-ebs-csi-driver-node-rg5jr\" (UID: \"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.080945 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.080745 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:10:00.080945 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.080769 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:10:00.080945 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.080787 2550 projected.go:196] Error preparing data for projected volume kube-api-access-pb97w for pod openshift-network-diagnostics/network-check-target-xctpc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:00.080945 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.080871 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w podName:537bb9e7-2c10-4fcf-a29c-352599686ffa nodeName:}" failed. No retries permitted until 2026-09-09 21:10:00.58085076 +0000 UTC m=+3.093587576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pb97w" (UniqueName: "kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w") pod "network-check-target-xctpc" (UID: "537bb9e7-2c10-4fcf-a29c-352599686ffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:00.081385 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.081004 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56hn5\" (UniqueName: \"kubernetes.io/projected/b3377582-07d8-484f-b42e-01b20a11dc48-kube-api-access-56hn5\") pod \"tuned-qk7vk\" (UID: \"b3377582-07d8-484f-b42e-01b20a11dc48\") " pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.084823 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.084800 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69lvv\" (UniqueName: \"kubernetes.io/projected/4b343c4d-2b12-4093-b6de-c4223e67f2bf-kube-api-access-69lvv\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:00.085757 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.085739 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmlvv\" (UniqueName: \"kubernetes.io/projected/9018849b-d1a1-4fcc-8854-4641e9a87cfe-kube-api-access-xmlvv\") pod \"iptables-alerter-dfjgf\" (UID: \"9018849b-d1a1-4fcc-8854-4641e9a87cfe\") " pod="openshift-network-operator/iptables-alerter-dfjgf" Sep 09 21:10:00.088162 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.087949 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqrgx\" (UniqueName: \"kubernetes.io/projected/4ab41251-7b22-458e-83c1-4dc1ed95611a-kube-api-access-nqrgx\") pod \"node-resolver-h8h2f\" (UID: \"4ab41251-7b22-458e-83c1-4dc1ed95611a\") " pod="openshift-dns/node-resolver-h8h2f" Sep 09 21:10:00.127639 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.127605 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:10:00.148577 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.148543 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:10:00.160009 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.159978 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rx56w" Sep 09 21:10:00.171001 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.170970 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9lzjh" Sep 09 21:10:00.180111 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.180085 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vth7c" Sep 09 21:10:00.190249 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.190227 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dfjgf" Sep 09 21:10:00.201372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.201345 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:10:00.209314 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.209293 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h8h2f" Sep 09 21:10:00.218356 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.218314 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" Sep 09 21:10:00.225424 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.225389 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" Sep 09 21:10:00.565030 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.564933 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:00.565030 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.564994 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:00.565288 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.565098 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:00.565288 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.565173 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs podName:4b343c4d-2b12-4093-b6de-c4223e67f2bf nodeName:}" failed. No retries permitted until 2026-09-09 21:10:01.565158012 +0000 UTC m=+4.077894810 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs") pod "network-metrics-daemon-9jn8c" (UID: "4b343c4d-2b12-4093-b6de-c4223e67f2bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:00.565288 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.565106 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:00.565288 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.565274 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret podName:4211bff4-82e4-4201-b854-de56fd0b7e14 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:01.565252953 +0000 UTC m=+4.077989770 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret") pod "global-pull-secret-syncer-kggbk" (UID: "4211bff4-82e4-4201-b854-de56fd0b7e14") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:00.665650 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.665610 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb97w\" (UniqueName: \"kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w\") pod \"network-check-target-xctpc\" (UID: \"537bb9e7-2c10-4fcf-a29c-352599686ffa\") " pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:00.665863 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.665830 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:10:00.665863 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.665867 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:10:00.666076 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.665889 2550 projected.go:196] Error preparing data for projected volume kube-api-access-pb97w for pod openshift-network-diagnostics/network-check-target-xctpc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:00.666076 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:00.665956 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w podName:537bb9e7-2c10-4fcf-a29c-352599686ffa nodeName:}" failed. No retries permitted until 2026-09-09 21:10:01.665935632 +0000 UTC m=+4.178672432 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pb97w" (UniqueName: "kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w") pod "network-check-target-xctpc" (UID: "537bb9e7-2c10-4fcf-a29c-352599686ffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:00.896733 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.896628 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 21:05:00 +0000 UTC" deadline="2028-02-08 02:53:39.5870024 +0000 UTC" Sep 09 21:10:00.896733 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:00.896675 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12389h43m38.690332665s" Sep 09 21:10:01.244905 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:01.244830 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2d7ff2_e63e_4fa5_91aa_23a373cdff4e.slice/crio-bb00f39c15d452df4008497f880a33028a3f53026adc672781feb2d9272053c8 WatchSource:0}: Error finding container bb00f39c15d452df4008497f880a33028a3f53026adc672781feb2d9272053c8: Status 404 returned error can't find the container with id bb00f39c15d452df4008497f880a33028a3f53026adc672781feb2d9272053c8 Sep 09 21:10:01.246405 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:01.246378 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9018849b_d1a1_4fcc_8854_4641e9a87cfe.slice/crio-d5ed4acf5336ee10d3ef52bcf5276550c768e2b396b5f5c4e25d4016056d6d65 WatchSource:0}: Error finding container d5ed4acf5336ee10d3ef52bcf5276550c768e2b396b5f5c4e25d4016056d6d65: Status 404 returned error can't find the container with id d5ed4acf5336ee10d3ef52bcf5276550c768e2b396b5f5c4e25d4016056d6d65 Sep 09 21:10:01.268770 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:01.268736 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a04424e_a63f_45bc_bd8b_53b3281e0982.slice/crio-ce7f57fbcca552a998a2e1838a0e12ec9137518948fe6c91423d8fdb3486a1d5 WatchSource:0}: Error finding container ce7f57fbcca552a998a2e1838a0e12ec9137518948fe6c91423d8fdb3486a1d5: Status 404 returned error can't find the container with id ce7f57fbcca552a998a2e1838a0e12ec9137518948fe6c91423d8fdb3486a1d5 Sep 09 21:10:01.269565 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:01.269534 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd3337d_7102_4a4b_826f_093466b1014d.slice/crio-ecff6758e286f1366ba69226988257872a19d4d0d0d501149dc58bf839040790 WatchSource:0}: Error finding container ecff6758e286f1366ba69226988257872a19d4d0d0d501149dc58bf839040790: Status 404 returned error can't find the container with id ecff6758e286f1366ba69226988257872a19d4d0d0d501149dc58bf839040790 Sep 09 21:10:01.272162 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:01.272128 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71354de2_a7c4_4336_9a2f_74cf7755fadb.slice/crio-25ee832e9104973e7b4b55df05c54e458c8de7eb6bdab6f173d78810979e2b04 WatchSource:0}: Error finding container 25ee832e9104973e7b4b55df05c54e458c8de7eb6bdab6f173d78810979e2b04: Status 404 returned error can't find the container with id 25ee832e9104973e7b4b55df05c54e458c8de7eb6bdab6f173d78810979e2b04 Sep 09 21:10:01.273236 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:01.273102 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3377582_07d8_484f_b42e_01b20a11dc48.slice/crio-adb5b94effd421c47f0dc70d3af0be269f1eecff170878b21aa5dc22cc07cb45 WatchSource:0}: Error finding container adb5b94effd421c47f0dc70d3af0be269f1eecff170878b21aa5dc22cc07cb45: Status 404 returned error can't find the container with id adb5b94effd421c47f0dc70d3af0be269f1eecff170878b21aa5dc22cc07cb45 Sep 09 21:10:01.274350 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:01.274321 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab41251_7b22_458e_83c1_4dc1ed95611a.slice/crio-0ce9ba69de342148371196e5299f506983cfbb1a84a298e33af86bbaa562e03c WatchSource:0}: Error finding container 0ce9ba69de342148371196e5299f506983cfbb1a84a298e33af86bbaa562e03c: Status 404 returned error can't find the container with id 0ce9ba69de342148371196e5299f506983cfbb1a84a298e33af86bbaa562e03c Sep 09 21:10:01.274911 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:01.274893 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda329dfed_fd34_425d_8372_fd0db6ac6bd7.slice/crio-e67d4524d5ba22183f1490b11cd663ad4d58c3d4f0b0f88eeb5970fc777b6ff0 WatchSource:0}: Error finding container e67d4524d5ba22183f1490b11cd663ad4d58c3d4f0b0f88eeb5970fc777b6ff0: Status 404 returned error can't find the container with id e67d4524d5ba22183f1490b11cd663ad4d58c3d4f0b0f88eeb5970fc777b6ff0 Sep 09 21:10:01.275939 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:01.275916 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8885188_4e52_4325_8d27_8f3ab4700578.slice/crio-d5e4fbe9354701009539b6148d852f885338f1bb0966bc0c72a3e3f26cc7c889 WatchSource:0}: Error finding container d5e4fbe9354701009539b6148d852f885338f1bb0966bc0c72a3e3f26cc7c889: Status 404 returned error can't find the container with id d5e4fbe9354701009539b6148d852f885338f1bb0966bc0c72a3e3f26cc7c889 Sep 09 21:10:01.574132 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:01.574084 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:01.574280 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:01.574164 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:01.574280 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:01.574242 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:01.574280 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:01.574250 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:01.574434 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:01.574313 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret podName:4211bff4-82e4-4201-b854-de56fd0b7e14 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:03.574296848 +0000 UTC m=+6.087033679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret") pod "global-pull-secret-syncer-kggbk" (UID: "4211bff4-82e4-4201-b854-de56fd0b7e14") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:01.574434 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:01.574329 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs podName:4b343c4d-2b12-4093-b6de-c4223e67f2bf nodeName:}" failed. No retries permitted until 2026-09-09 21:10:03.574321349 +0000 UTC m=+6.087058148 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs") pod "network-metrics-daemon-9jn8c" (UID: "4b343c4d-2b12-4093-b6de-c4223e67f2bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:01.675121 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:01.675081 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb97w\" (UniqueName: \"kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w\") pod \"network-check-target-xctpc\" (UID: \"537bb9e7-2c10-4fcf-a29c-352599686ffa\") " pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:01.675324 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:01.675215 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:10:01.675324 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:01.675230 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:10:01.675324 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:01.675239 2550 projected.go:196] Error preparing data for projected volume kube-api-access-pb97w for pod openshift-network-diagnostics/network-check-target-xctpc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:01.675324 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:01.675292 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w podName:537bb9e7-2c10-4fcf-a29c-352599686ffa nodeName:}" failed. No retries permitted until 2026-09-09 21:10:03.675279023 +0000 UTC m=+6.188015821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pb97w" (UniqueName: "kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w") pod "network-check-target-xctpc" (UID: "537bb9e7-2c10-4fcf-a29c-352599686ffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:01.897421 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:01.897291 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 21:05:00 +0000 UTC" deadline="2028-04-27 11:08:04.075201479 +0000 UTC" Sep 09 21:10:01.897421 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:01.897351 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14293h58m2.177852921s" Sep 09 21:10:01.985177 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:01.985141 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:01.985177 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:01.985161 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:01.985460 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:01.985162 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:01.985460 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:01.985280 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:01.985460 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:01.985411 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:01.985621 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:01.985523 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:01.996646 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:01.996611 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dfjgf" event={"ID":"9018849b-d1a1-4fcc-8854-4641e9a87cfe","Type":"ContainerStarted","Data":"d5ed4acf5336ee10d3ef52bcf5276550c768e2b396b5f5c4e25d4016056d6d65"} Sep 09 21:10:01.998830 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:01.998792 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-60.ec2.internal" event={"ID":"d53d5013d65214a919bdfb5b967b3585","Type":"ContainerStarted","Data":"59b62cbc8228040cc995cfe574634214b2ca7934f2ef78de1438035b3a820f66"} Sep 09 21:10:02.000180 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:02.000157 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qrjzg" event={"ID":"e8885188-4e52-4325-8d27-8f3ab4700578","Type":"ContainerStarted","Data":"d5e4fbe9354701009539b6148d852f885338f1bb0966bc0c72a3e3f26cc7c889"} Sep 09 21:10:02.001475 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:02.001445 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9lzjh" event={"ID":"a329dfed-fd34-425d-8372-fd0db6ac6bd7","Type":"ContainerStarted","Data":"e67d4524d5ba22183f1490b11cd663ad4d58c3d4f0b0f88eeb5970fc777b6ff0"} Sep 09 21:10:02.002672 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:02.002646 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h8h2f" event={"ID":"4ab41251-7b22-458e-83c1-4dc1ed95611a","Type":"ContainerStarted","Data":"0ce9ba69de342148371196e5299f506983cfbb1a84a298e33af86bbaa562e03c"} Sep 09 21:10:02.003736 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:02.003706 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rx56w" event={"ID":"fdd3337d-7102-4a4b-826f-093466b1014d","Type":"ContainerStarted","Data":"ecff6758e286f1366ba69226988257872a19d4d0d0d501149dc58bf839040790"} Sep 09 21:10:02.004844 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:02.004822 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" event={"ID":"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e","Type":"ContainerStarted","Data":"bb00f39c15d452df4008497f880a33028a3f53026adc672781feb2d9272053c8"} Sep 09 21:10:02.006588 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:02.006566 2550 generic.go:356] "Generic (PLEG): container finished" podID="ee47118de8909838405186e61d51474b" containerID="a71cb8ecf70b9a037624e10fdef234a3c17e3f7e1551c9eb2e58dd1d979ff369" exitCode=0 Sep 09 21:10:02.006676 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:02.006606 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" event={"ID":"ee47118de8909838405186e61d51474b","Type":"ContainerDied","Data":"a71cb8ecf70b9a037624e10fdef234a3c17e3f7e1551c9eb2e58dd1d979ff369"} Sep 09 21:10:02.007917 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:02.007889 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" event={"ID":"b3377582-07d8-484f-b42e-01b20a11dc48","Type":"ContainerStarted","Data":"adb5b94effd421c47f0dc70d3af0be269f1eecff170878b21aa5dc22cc07cb45"} Sep 09 21:10:02.009246 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:02.009217 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" event={"ID":"71354de2-a7c4-4336-9a2f-74cf7755fadb","Type":"ContainerStarted","Data":"25ee832e9104973e7b4b55df05c54e458c8de7eb6bdab6f173d78810979e2b04"} Sep 09 21:10:02.010373 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:02.010352 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vth7c" event={"ID":"0a04424e-a63f-45bc-bd8b-53b3281e0982","Type":"ContainerStarted","Data":"ce7f57fbcca552a998a2e1838a0e12ec9137518948fe6c91423d8fdb3486a1d5"} Sep 09 21:10:02.015305 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:02.015266 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-60.ec2.internal" podStartSLOduration=3.015253821 podStartE2EDuration="3.015253821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:02.014708453 +0000 UTC m=+4.527445299" watchObservedRunningTime="2026-09-09 21:10:02.015253821 +0000 UTC m=+4.527990643" Sep 09 21:10:03.027769 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:03.027469 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" event={"ID":"ee47118de8909838405186e61d51474b","Type":"ContainerStarted","Data":"35f32ab3b0daa9975535daaca1dd39fcafc41062c69ff6c7eade022cb34df647"} Sep 09 21:10:03.046777 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:03.046711 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-60.ec2.internal" podStartSLOduration=4.046686758 podStartE2EDuration="4.046686758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:03.04622816 +0000 UTC m=+5.558965011" watchObservedRunningTime="2026-09-09 21:10:03.046686758 +0000 UTC m=+5.559423596" Sep 09 21:10:03.593287 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:03.592421 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:03.593287 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:03.592493 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:03.593287 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:03.592654 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:03.593287 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:03.592722 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret podName:4211bff4-82e4-4201-b854-de56fd0b7e14 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:07.592703287 +0000 UTC m=+10.105440085 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret") pod "global-pull-secret-syncer-kggbk" (UID: "4211bff4-82e4-4201-b854-de56fd0b7e14") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:03.593287 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:03.593141 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:03.593287 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:03.593190 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs podName:4b343c4d-2b12-4093-b6de-c4223e67f2bf nodeName:}" failed. No retries permitted until 2026-09-09 21:10:07.593175248 +0000 UTC m=+10.105912061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs") pod "network-metrics-daemon-9jn8c" (UID: "4b343c4d-2b12-4093-b6de-c4223e67f2bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:03.693785 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:03.693746 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb97w\" (UniqueName: \"kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w\") pod \"network-check-target-xctpc\" (UID: \"537bb9e7-2c10-4fcf-a29c-352599686ffa\") " pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:03.693978 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:03.693952 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:10:03.694057 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:03.693987 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:10:03.694057 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:03.694002 2550 projected.go:196] Error preparing data for projected volume kube-api-access-pb97w for pod openshift-network-diagnostics/network-check-target-xctpc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:03.694163 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:03.694070 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w podName:537bb9e7-2c10-4fcf-a29c-352599686ffa nodeName:}" failed. No retries permitted until 2026-09-09 21:10:07.694048237 +0000 UTC m=+10.206785038 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pb97w" (UniqueName: "kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w") pod "network-check-target-xctpc" (UID: "537bb9e7-2c10-4fcf-a29c-352599686ffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:03.985236 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:03.985138 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:03.985236 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:03.985181 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:03.985236 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:03.985218 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:03.985629 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:03.985361 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:03.985629 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:03.985513 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:03.985753 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:03.985648 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:05.985701 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:05.985611 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:05.985701 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:05.985646 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:05.986226 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:05.985756 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:05.986226 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:05.986125 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:05.986226 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:05.986208 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:05.986394 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:05.986297 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:07.636504 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:07.635736 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:07.636504 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:07.635820 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:07.636504 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:07.635967 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:07.636504 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:07.636029 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs podName:4b343c4d-2b12-4093-b6de-c4223e67f2bf nodeName:}" failed. No retries permitted until 2026-09-09 21:10:15.636008152 +0000 UTC m=+18.148744952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs") pod "network-metrics-daemon-9jn8c" (UID: "4b343c4d-2b12-4093-b6de-c4223e67f2bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:07.636504 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:07.636413 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:07.636504 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:07.636458 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret podName:4211bff4-82e4-4201-b854-de56fd0b7e14 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:15.636444388 +0000 UTC m=+18.149181204 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret") pod "global-pull-secret-syncer-kggbk" (UID: "4211bff4-82e4-4201-b854-de56fd0b7e14") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:07.736465 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:07.736411 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb97w\" (UniqueName: \"kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w\") pod \"network-check-target-xctpc\" (UID: \"537bb9e7-2c10-4fcf-a29c-352599686ffa\") " pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:07.736648 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:07.736630 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:10:07.736745 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:07.736652 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:10:07.736745 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:07.736660 2550 projected.go:196] Error preparing data for projected volume kube-api-access-pb97w for pod openshift-network-diagnostics/network-check-target-xctpc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:07.736745 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:07.736710 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w podName:537bb9e7-2c10-4fcf-a29c-352599686ffa nodeName:}" failed. No retries permitted until 2026-09-09 21:10:15.73669363 +0000 UTC m=+18.249430431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pb97w" (UniqueName: "kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w") pod "network-check-target-xctpc" (UID: "537bb9e7-2c10-4fcf-a29c-352599686ffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:07.986378 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:07.986285 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:07.986532 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:07.986435 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:07.987527 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:07.987502 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:07.987634 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:07.987609 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:07.987691 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:07.987660 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:07.987846 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:07.987737 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:09.985086 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:09.985054 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:09.985543 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:09.985060 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:09.985543 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:09.985182 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:09.985543 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:09.985206 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:09.985543 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:09.985296 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:09.985961 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:09.985813 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:11.984697 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:11.984609 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:11.985193 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:11.984610 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:11.985193 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:11.984763 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:11.985193 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:11.984741 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:11.985193 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:11.984862 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:11.985193 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:11.984933 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:13.984728 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:13.984686 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:13.985185 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:13.984686 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:13.985185 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:13.984839 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:13.985185 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:13.984686 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:13.985185 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:13.984926 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:13.985185 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:13.984976 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:15.698435 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:15.698400 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:15.699014 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:15.698475 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:15.699014 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:15.698589 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:15.699014 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:15.698590 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:15.699014 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:15.698662 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs podName:4b343c4d-2b12-4093-b6de-c4223e67f2bf nodeName:}" failed. No retries permitted until 2026-09-09 21:10:31.698641874 +0000 UTC m=+34.211378684 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs") pod "network-metrics-daemon-9jn8c" (UID: "4b343c4d-2b12-4093-b6de-c4223e67f2bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:15.699014 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:15.698682 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret podName:4211bff4-82e4-4201-b854-de56fd0b7e14 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:31.698672772 +0000 UTC m=+34.211409569 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret") pod "global-pull-secret-syncer-kggbk" (UID: "4211bff4-82e4-4201-b854-de56fd0b7e14") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:15.799520 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:15.799483 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb97w\" (UniqueName: \"kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w\") pod \"network-check-target-xctpc\" (UID: \"537bb9e7-2c10-4fcf-a29c-352599686ffa\") " pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:15.799696 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:15.799609 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:10:15.799696 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:15.799623 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:10:15.799696 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:15.799632 2550 projected.go:196] Error preparing data for projected volume kube-api-access-pb97w for pod openshift-network-diagnostics/network-check-target-xctpc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:15.799696 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:15.799681 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w podName:537bb9e7-2c10-4fcf-a29c-352599686ffa nodeName:}" failed. No retries permitted until 2026-09-09 21:10:31.79966853 +0000 UTC m=+34.312405327 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pb97w" (UniqueName: "kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w") pod "network-check-target-xctpc" (UID: "537bb9e7-2c10-4fcf-a29c-352599686ffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:15.985225 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:15.985140 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:15.985225 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:15.985156 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:15.985465 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:15.985289 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:15.985465 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:15.985378 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:15.985465 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:15.985370 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:15.985465 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:15.985448 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:17.985033 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:17.984992 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:17.985528 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:17.985107 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:17.985528 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:17.985225 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:17.985528 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:17.985117 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:17.985528 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:17.985291 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:17.985528 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:17.985387 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:19.984725 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:19.984536 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:19.985634 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:19.984536 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:19.985634 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:19.984808 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:19.985634 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:19.984870 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:19.985634 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:19.984561 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:19.985634 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:19.984950 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:20.073260 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.073013 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qrjzg" event={"ID":"e8885188-4e52-4325-8d27-8f3ab4700578","Type":"ContainerStarted","Data":"44d16eaadbaf786bb64cf688d5afff83fa67ca82e29785748178381883528c60"} Sep 09 21:10:20.074721 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.074638 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9lzjh" event={"ID":"a329dfed-fd34-425d-8372-fd0db6ac6bd7","Type":"ContainerStarted","Data":"a7ce8fa2e657f219fb08c513a2248dd40da09d365fd4e7fda18f5ab9c91821f3"} Sep 09 21:10:20.076506 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.076476 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h8h2f" event={"ID":"4ab41251-7b22-458e-83c1-4dc1ed95611a","Type":"ContainerStarted","Data":"faebe118594b3676a46e17b35ce0e645c8aa3034ba9ca8f68b5cec488458cfe3"} Sep 09 21:10:20.078115 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.078083 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rx56w" event={"ID":"fdd3337d-7102-4a4b-826f-093466b1014d","Type":"ContainerStarted","Data":"6e3305428d4d22ce6348a3acf6744fe589af4bf59cb575646ccc1290c664814f"} Sep 09 21:10:20.079747 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.079723 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" event={"ID":"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e","Type":"ContainerStarted","Data":"d61e91fd22f8642da4404ed0eee9ad6bbd5311a131b59c56cb4874c22478bb98"} Sep 09 21:10:20.081330 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.081293 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" event={"ID":"b3377582-07d8-484f-b42e-01b20a11dc48","Type":"ContainerStarted","Data":"05758617f05ba1fb6a2a73e334c775d9571bfbb7b151ea444d52d6339644f87a"} Sep 09 21:10:20.084277 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.084251 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" event={"ID":"71354de2-a7c4-4336-9a2f-74cf7755fadb","Type":"ContainerStarted","Data":"5d27177e183c47718218b08c94a08e4d90405309ed4dd82a90e0fd5abeeb55c6"} Sep 09 21:10:20.084401 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.084349 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" event={"ID":"71354de2-a7c4-4336-9a2f-74cf7755fadb","Type":"ContainerStarted","Data":"cfa493c7b87ddb2bf46ce1a5554b0f8cf1c2ac96a03be8cc6420d169e2a611e2"} Sep 09 21:10:20.084401 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.084394 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" event={"ID":"71354de2-a7c4-4336-9a2f-74cf7755fadb","Type":"ContainerStarted","Data":"2332b5c2e2e426ce8cce8d4df2f887bf64726f27875c1515f22cbdfd4892223e"} Sep 09 21:10:20.085936 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.085884 2550 generic.go:356] "Generic (PLEG): container finished" podID="0a04424e-a63f-45bc-bd8b-53b3281e0982" containerID="04619fa228b967a5908fdafcd2a625d7cd09b2398b3f1d40c5f158d9288dc365" exitCode=0 Sep 09 21:10:20.086051 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.085949 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vth7c" event={"ID":"0a04424e-a63f-45bc-bd8b-53b3281e0982","Type":"ContainerDied","Data":"04619fa228b967a5908fdafcd2a625d7cd09b2398b3f1d40c5f158d9288dc365"} Sep 09 21:10:20.089783 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.089739 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qrjzg" podStartSLOduration=10.993181172 podStartE2EDuration="22.089726042s" totalImagesPullingTime="11.09654487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:20.088979419 +0000 UTC m=+22.601716254" watchObservedRunningTime="2026-09-09 21:10:20.089726042 +0000 UTC m=+22.602462855" Sep 09 21:10:20.125025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.124981 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rx56w" podStartSLOduration=3.715035881 podStartE2EDuration="22.124964486s" totalImagesPullingTime="18.409928605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:20.105512434 +0000 UTC m=+22.618249270" watchObservedRunningTime="2026-09-09 21:10:20.124964486 +0000 UTC m=+22.637701305" Sep 09 21:10:20.125150 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.125133 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9lzjh" podStartSLOduration=3.662620666 podStartE2EDuration="22.125129358s" totalImagesPullingTime="18.462508692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:20.124447711 +0000 UTC m=+22.637184541" watchObservedRunningTime="2026-09-09 21:10:20.125129358 +0000 UTC m=+22.637866176" Sep 09 21:10:20.159151 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.159098 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-h8h2f" podStartSLOduration=3.754308885 podStartE2EDuration="22.1590765s" totalImagesPullingTime="18.404767615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:20.140275494 +0000 UTC m=+22.653012346" watchObservedRunningTime="2026-09-09 21:10:20.1590765 +0000 UTC m=+22.671813322" Sep 09 21:10:20.180810 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:20.180714 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qk7vk" podStartSLOduration=3.7742548940000002 podStartE2EDuration="22.180698509s" totalImagesPullingTime="18.406443615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:20.159161034 +0000 UTC m=+22.671897865" watchObservedRunningTime="2026-09-09 21:10:20.180698509 +0000 UTC m=+22.693435338" Sep 09 21:10:21.092351 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:21.092274 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" event={"ID":"71354de2-a7c4-4336-9a2f-74cf7755fadb","Type":"ContainerStarted","Data":"389d7baddd8cec7540443ba232b8a7966650f1c473976d14d19361a86c603d14"} Sep 09 21:10:21.092807 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:21.092484 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" event={"ID":"71354de2-a7c4-4336-9a2f-74cf7755fadb","Type":"ContainerStarted","Data":"fad348c9538d13d6dbf6a6c5748bd89ea46c9e059bfc027492e5b8408780504b"} Sep 09 21:10:21.092807 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:21.092546 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" event={"ID":"71354de2-a7c4-4336-9a2f-74cf7755fadb","Type":"ContainerStarted","Data":"b763b82259fde0ce779645a5584072bd7c6f520ff8f624fa780245e9f76a74fb"} Sep 09 21:10:21.233122 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:21.233083 2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 21:10:21.920379 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:21.920235 2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T21:10:21.233111782Z","UUID":"e0bec9a7-465f-45b3-9356-ce0c232e176c","Handler":null,"Name":"","Endpoint":""} Sep 09 21:10:21.922040 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:21.922015 2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 21:10:21.922164 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:21.922052 2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 21:10:21.985100 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:21.984592 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:21.985100 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:21.984607 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:21.985100 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:21.984717 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:21.985100 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:21.984765 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:21.985100 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:21.984866 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:21.985100 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:21.984966 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:22.098150 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:22.098111 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" event={"ID":"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e","Type":"ContainerStarted","Data":"66b746d6fa51cb28052a58de4ba3b9f87ccb9c7692ceb0b52c9c1abf7359991e"} Sep 09 21:10:22.099868 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:22.099838 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dfjgf" event={"ID":"9018849b-d1a1-4fcc-8854-4641e9a87cfe","Type":"ContainerStarted","Data":"f575ae25e62f612daaa51553403b18b9da07e8425185d1a7691ed6ebcd81d874"} Sep 09 21:10:23.105704 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:23.105667 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" event={"ID":"71354de2-a7c4-4336-9a2f-74cf7755fadb","Type":"ContainerStarted","Data":"a0a721214d7380aad5adac0a3a591e2528bdcbe56616450f9a0defb9812238f0"} Sep 09 21:10:23.107917 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:23.107890 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" event={"ID":"8e2d7ff2-e63e-4fa5-91aa-23a373cdff4e","Type":"ContainerStarted","Data":"f879394215aeaf55f05d256bbd6faafdb9f4b23f46c8962c368d4f99e17d5879"} Sep 09 21:10:23.126741 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:23.126688 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dfjgf" podStartSLOduration=6.711440527 podStartE2EDuration="25.126669174s" totalImagesPullingTime="18.415228647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:22.126139578 +0000 UTC m=+24.638876399" watchObservedRunningTime="2026-09-09 21:10:23.126669174 +0000 UTC m=+25.639405992" Sep 09 21:10:23.127017 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:23.126994 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg5jr" podStartSLOduration=4.658095673 podStartE2EDuration="25.126987115s" totalImagesPullingTime="20.468891442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:23.126352173 +0000 UTC m=+25.639089000" watchObservedRunningTime="2026-09-09 21:10:23.126987115 +0000 UTC m=+25.639723931" Sep 09 21:10:23.985368 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:23.985312 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:23.985550 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:23.985344 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:23.985550 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:23.985453 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:23.985550 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:23.985337 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:23.985550 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:23.985517 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:23.985720 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:23.985594 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:24.908083 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:24.907888 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:10:24.908720 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:24.908481 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:10:25.116232 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.116147 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" event={"ID":"71354de2-a7c4-4336-9a2f-74cf7755fadb","Type":"ContainerStarted","Data":"be33f7a498c3ce1bc8e39fcc75770be8bed73e2eeac7d6be498703c3b02a8369"} Sep 09 21:10:25.116469 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.116449 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:10:25.116606 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.116577 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:10:25.116713 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.116633 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:10:25.118720 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.118699 2550 generic.go:356] "Generic (PLEG): container finished" podID="0a04424e-a63f-45bc-bd8b-53b3281e0982" containerID="c9a020e4ef43f1f900b35f644b9a1490b00b68c7945a23ef7851927646bc33c3" exitCode=0 Sep 09 21:10:25.118831 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.118765 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vth7c" event={"ID":"0a04424e-a63f-45bc-bd8b-53b3281e0982","Type":"ContainerDied","Data":"c9a020e4ef43f1f900b35f644b9a1490b00b68c7945a23ef7851927646bc33c3"} Sep 09 21:10:25.118946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.118933 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:10:25.119532 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.119513 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qrjzg" Sep 09 21:10:25.132769 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.132746 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:10:25.132916 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.132851 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:10:25.147265 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.147220 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" podStartSLOduration=8.681175402000001 podStartE2EDuration="27.147201679s" totalImagesPullingTime="18.466026277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:25.146829351 +0000 UTC m=+27.659566170" watchObservedRunningTime="2026-09-09 21:10:25.147201679 +0000 UTC m=+27.659938497" Sep 09 21:10:25.984558 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.984520 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:25.984558 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.984553 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:25.985124 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:25.984659 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:25.985124 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:25.984722 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:25.985124 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:25.984826 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:25.985124 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:25.984935 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:26.435720 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:26.435513 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9jn8c"] Sep 09 21:10:26.435934 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:26.435816 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:26.435934 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:26.435911 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:26.436243 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:26.436215 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kggbk"] Sep 09 21:10:26.436378 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:26.436301 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:26.436454 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:26.436412 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:26.448951 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:26.448925 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xctpc"] Sep 09 21:10:26.449129 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:26.449015 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:26.449129 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:26.449092 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:27.125345 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:27.125298 2550 generic.go:356] "Generic (PLEG): container finished" podID="0a04424e-a63f-45bc-bd8b-53b3281e0982" containerID="11740b1e687be39f3da4288c22f8e70a270c7764bbb2596c9b99ed26b2f2e1ac" exitCode=0 Sep 09 21:10:27.125726 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:27.125377 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vth7c" event={"ID":"0a04424e-a63f-45bc-bd8b-53b3281e0982","Type":"ContainerDied","Data":"11740b1e687be39f3da4288c22f8e70a270c7764bbb2596c9b99ed26b2f2e1ac"} Sep 09 21:10:27.986370 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:27.986313 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:27.986370 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:27.986366 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:27.986588 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:27.986463 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:27.986588 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:27.986474 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:27.986588 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:27.986539 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:27.986684 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:27.986592 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:29.132141 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:29.132111 2550 generic.go:356] "Generic (PLEG): container finished" podID="0a04424e-a63f-45bc-bd8b-53b3281e0982" containerID="adb11a8c5800776328a16b6518c5bc618b6f56cd4bf1d6b792105ad85ce05555" exitCode=0 Sep 09 21:10:29.132528 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:29.132149 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vth7c" event={"ID":"0a04424e-a63f-45bc-bd8b-53b3281e0982","Type":"ContainerDied","Data":"adb11a8c5800776328a16b6518c5bc618b6f56cd4bf1d6b792105ad85ce05555"} Sep 09 21:10:29.985572 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:29.985366 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:29.985763 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:29.985366 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:29.985763 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:29.985682 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:29.985763 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:29.985367 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:29.985763 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:29.985748 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:29.985919 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:29.985830 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:31.714581 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:31.714544 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:31.714936 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:31.714607 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:31.714936 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:31.714694 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:31.714936 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:31.714698 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:31.714936 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:31.714744 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs podName:4b343c4d-2b12-4093-b6de-c4223e67f2bf nodeName:}" failed. No retries permitted until 2026-09-09 21:11:03.714730707 +0000 UTC m=+66.227467504 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs") pod "network-metrics-daemon-9jn8c" (UID: "4b343c4d-2b12-4093-b6de-c4223e67f2bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:31.714936 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:31.714758 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret podName:4211bff4-82e4-4201-b854-de56fd0b7e14 nodeName:}" failed. No retries permitted until 2026-09-09 21:11:03.714751946 +0000 UTC m=+66.227488742 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret") pod "global-pull-secret-syncer-kggbk" (UID: "4211bff4-82e4-4201-b854-de56fd0b7e14") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:31.815306 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:31.815264 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb97w\" (UniqueName: \"kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w\") pod \"network-check-target-xctpc\" (UID: \"537bb9e7-2c10-4fcf-a29c-352599686ffa\") " pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:31.815507 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:31.815468 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:10:31.815507 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:31.815492 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:10:31.815507 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:31.815504 2550 projected.go:196] Error preparing data for projected volume kube-api-access-pb97w for pod openshift-network-diagnostics/network-check-target-xctpc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:31.815654 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:31.815566 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w podName:537bb9e7-2c10-4fcf-a29c-352599686ffa nodeName:}" failed. No retries permitted until 2026-09-09 21:11:03.815545425 +0000 UTC m=+66.328282242 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pb97w" (UniqueName: "kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w") pod "network-check-target-xctpc" (UID: "537bb9e7-2c10-4fcf-a29c-352599686ffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:31.987490 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:31.987412 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:31.987644 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:31.987531 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jn8c" podUID="4b343c4d-2b12-4093-b6de-c4223e67f2bf" Sep 09 21:10:31.987922 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:31.987903 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:31.988012 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:31.987991 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xctpc" podUID="537bb9e7-2c10-4fcf-a29c-352599686ffa" Sep 09 21:10:31.988076 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:31.988062 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:31.988165 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:31.988145 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kggbk" podUID="4211bff4-82e4-4201-b854-de56fd0b7e14" Sep 09 21:10:33.768903 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.768865 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-60.ec2.internal" event="NodeReady" Sep 09 21:10:33.769352 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.768992 2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 21:10:33.807449 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.807405 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-864846d867-ptrds"] Sep 09 21:10:33.809352 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.809313 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:33.811639 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.811617 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 09 21:10:33.811753 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.811688 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xkzkp\"" Sep 09 21:10:33.811946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.811829 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 09 21:10:33.813189 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.813174 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 09 21:10:33.821540 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.821500 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jcd2g"] Sep 09 21:10:33.823541 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.823516 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 09 21:10:33.824538 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.824519 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:33.828747 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.828697 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jq8z4"] Sep 09 21:10:33.831668 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.831644 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-864846d867-ptrds"] Sep 09 21:10:33.831763 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.831742 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jq8z4" Sep 09 21:10:33.835603 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.835571 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 21:10:33.835700 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.835617 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 21:10:33.835700 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.835638 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 21:10:33.835700 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.835644 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 21:10:33.835700 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.835617 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7flhb\"" Sep 09 21:10:33.835876 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.835838 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 21:10:33.836008 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.835996 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v6fqv\"" Sep 09 21:10:33.843210 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.843186 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jq8z4"] Sep 09 21:10:33.851022 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.850993 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jcd2g"] Sep 09 21:10:33.930805 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.930767 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73b7970-9da3-44fb-bd20-efcc68505662-tmp-dir\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:33.930984 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.930818 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eca72d7b-7575-4cec-881d-d1d9ae20f872-ca-trust-extracted\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:33.930984 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.930835 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gshcw\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-kube-api-access-gshcw\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:33.930984 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.930904 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d73b7970-9da3-44fb-bd20-efcc68505662-config-volume\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:33.930984 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.930932 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66hnq\" (UniqueName: \"kubernetes.io/projected/3ac31950-bf71-41ae-97ce-a45f4533baf0-kube-api-access-66hnq\") pod \"ingress-canary-jq8z4\" (UID: \"3ac31950-bf71-41ae-97ce-a45f4533baf0\") " pod="openshift-ingress-canary/ingress-canary-jq8z4" Sep 09 21:10:33.930984 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.930952 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-certificates\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:33.930984 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.930971 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-image-registry-private-configuration\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:33.931196 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.930987 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:33.931196 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.931004 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ac31950-bf71-41ae-97ce-a45f4533baf0-cert\") pod \"ingress-canary-jq8z4\" (UID: \"3ac31950-bf71-41ae-97ce-a45f4533baf0\") " pod="openshift-ingress-canary/ingress-canary-jq8z4" Sep 09 21:10:33.931196 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.931021 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-installation-pull-secrets\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:33.931196 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.931045 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-bound-sa-token\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:33.931196 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.931073 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-trusted-ca\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:33.931196 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.931098 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkg6t\" (UniqueName: \"kubernetes.io/projected/d73b7970-9da3-44fb-bd20-efcc68505662-kube-api-access-hkg6t\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:33.931196 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.931141 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:33.985105 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.985073 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:10:33.985105 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.985104 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:10:33.985286 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.985073 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:10:33.987732 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.987706 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 21:10:33.987732 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.987727 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ck66v\"" Sep 09 21:10:33.987732 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.987734 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 21:10:33.988054 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.988033 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 21:10:33.988195 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.988178 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7j9l\"" Sep 09 21:10:33.988355 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:33.988338 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 21:10:34.031633 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031552 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eca72d7b-7575-4cec-881d-d1d9ae20f872-ca-trust-extracted\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.031633 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031583 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gshcw\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-kube-api-access-gshcw\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.031633 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031606 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d73b7970-9da3-44fb-bd20-efcc68505662-config-volume\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:34.031633 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031623 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66hnq\" (UniqueName: \"kubernetes.io/projected/3ac31950-bf71-41ae-97ce-a45f4533baf0-kube-api-access-66hnq\") pod \"ingress-canary-jq8z4\" (UID: \"3ac31950-bf71-41ae-97ce-a45f4533baf0\") " pod="openshift-ingress-canary/ingress-canary-jq8z4" Sep 09 21:10:34.031946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031640 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-certificates\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.031946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031766 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-image-registry-private-configuration\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.031946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031806 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:34.031946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031839 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ac31950-bf71-41ae-97ce-a45f4533baf0-cert\") pod \"ingress-canary-jq8z4\" (UID: \"3ac31950-bf71-41ae-97ce-a45f4533baf0\") " pod="openshift-ingress-canary/ingress-canary-jq8z4" Sep 09 21:10:34.031946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031863 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-installation-pull-secrets\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.031946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031892 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-bound-sa-token\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.031946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031934 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-trusted-ca\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.032299 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.031974 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkg6t\" (UniqueName: \"kubernetes.io/projected/d73b7970-9da3-44fb-bd20-efcc68505662-kube-api-access-hkg6t\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:34.032299 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.032006 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.032299 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.032053 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eca72d7b-7575-4cec-881d-d1d9ae20f872-ca-trust-extracted\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.032299 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.032060 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73b7970-9da3-44fb-bd20-efcc68505662-tmp-dir\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:34.032299 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.031934 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:10:34.032299 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.032154 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:10:34.032299 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.032164 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864846d867-ptrds: secret "image-registry-tls" not found Sep 09 21:10:34.032299 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.032224 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d73b7970-9da3-44fb-bd20-efcc68505662-config-volume\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:34.032299 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.032288 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-certificates\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.032798 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.032368 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls podName:eca72d7b-7575-4cec-881d-d1d9ae20f872 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:34.532314747 +0000 UTC m=+37.045051544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls") pod "image-registry-864846d867-ptrds" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872") : secret "image-registry-tls" not found Sep 09 21:10:34.032798 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.032390 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls podName:d73b7970-9da3-44fb-bd20-efcc68505662 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:34.532380144 +0000 UTC m=+37.045116943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls") pod "dns-default-jcd2g" (UID: "d73b7970-9da3-44fb-bd20-efcc68505662") : secret "dns-default-metrics-tls" not found Sep 09 21:10:34.032798 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.032564 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73b7970-9da3-44fb-bd20-efcc68505662-tmp-dir\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:34.033136 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.033112 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-trusted-ca\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.035989 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.035968 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-installation-pull-secrets\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.036077 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.035991 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-image-registry-private-configuration\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.036077 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.036049 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ac31950-bf71-41ae-97ce-a45f4533baf0-cert\") pod \"ingress-canary-jq8z4\" (UID: \"3ac31950-bf71-41ae-97ce-a45f4533baf0\") " pod="openshift-ingress-canary/ingress-canary-jq8z4" Sep 09 21:10:34.045273 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.045247 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66hnq\" (UniqueName: \"kubernetes.io/projected/3ac31950-bf71-41ae-97ce-a45f4533baf0-kube-api-access-66hnq\") pod \"ingress-canary-jq8z4\" (UID: \"3ac31950-bf71-41ae-97ce-a45f4533baf0\") " pod="openshift-ingress-canary/ingress-canary-jq8z4" Sep 09 21:10:34.045644 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.045621 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkg6t\" (UniqueName: \"kubernetes.io/projected/d73b7970-9da3-44fb-bd20-efcc68505662-kube-api-access-hkg6t\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:34.046568 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.046549 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gshcw\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-kube-api-access-gshcw\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.046786 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.046769 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-bound-sa-token\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.141297 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.141263 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jq8z4" Sep 09 21:10:34.147074 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.147048 2550 generic.go:356] "Generic (PLEG): container finished" podID="0a04424e-a63f-45bc-bd8b-53b3281e0982" containerID="0e762a4026ec219da16f32787ded2938ed4c00e7e061150be704627741cd2114" exitCode=0 Sep 09 21:10:34.147160 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.147086 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vth7c" event={"ID":"0a04424e-a63f-45bc-bd8b-53b3281e0982","Type":"ContainerDied","Data":"0e762a4026ec219da16f32787ded2938ed4c00e7e061150be704627741cd2114"} Sep 09 21:10:34.324108 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.323840 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jvgd8"] Sep 09 21:10:34.326183 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.326167 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.329301 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.329281 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 21:10:34.329618 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.329603 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 21:10:34.329927 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.329910 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 21:10:34.330009 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.329958 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 21:10:34.330163 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.330149 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wl9jm\"" Sep 09 21:10:34.341994 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.341970 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jvgd8"] Sep 09 21:10:34.433091 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.433062 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jq8z4"] Sep 09 21:10:34.435420 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.435400 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/78ce6eba-2e3e-4a0e-96ad-13be205f151e-data-volume\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.435506 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.435437 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.435506 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.435496 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmbpv\" (UniqueName: \"kubernetes.io/projected/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-api-access-bmbpv\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.435588 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.435535 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.435671 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.435651 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/78ce6eba-2e3e-4a0e-96ad-13be205f151e-crio-socket\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.436336 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:34.436298 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac31950_bf71_41ae_97ce_a45f4533baf0.slice/crio-eeffabcf852b311fa68e65673bfe2f30718b769dab68ae5cecb0f01ff98590ee WatchSource:0}: Error finding container eeffabcf852b311fa68e65673bfe2f30718b769dab68ae5cecb0f01ff98590ee: Status 404 returned error can't find the container with id eeffabcf852b311fa68e65673bfe2f30718b769dab68ae5cecb0f01ff98590ee Sep 09 21:10:34.536157 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.536122 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmbpv\" (UniqueName: \"kubernetes.io/projected/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-api-access-bmbpv\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.536434 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.536176 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.536434 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.536200 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:34.536434 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.536269 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:10:34.536434 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.536286 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:10:34.536434 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.536298 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864846d867-ptrds: secret "image-registry-tls" not found Sep 09 21:10:34.536434 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.536364 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls podName:eca72d7b-7575-4cec-881d-d1d9ae20f872 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:35.536348618 +0000 UTC m=+38.049085414 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls") pod "image-registry-864846d867-ptrds" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872") : secret "image-registry-tls" not found Sep 09 21:10:34.536434 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.536389 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/78ce6eba-2e3e-4a0e-96ad-13be205f151e-crio-socket\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.536434 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.536400 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm podName:78ce6eba-2e3e-4a0e-96ad-13be205f151e nodeName:}" failed. No retries permitted until 2026-09-09 21:10:35.03638103 +0000 UTC m=+37.549117845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jvgd8" (UID: "78ce6eba-2e3e-4a0e-96ad-13be205f151e") : configmap "kube-rbac-proxy" not found Sep 09 21:10:34.536743 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.536460 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/78ce6eba-2e3e-4a0e-96ad-13be205f151e-data-volume\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.536743 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.536483 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.536743 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.536502 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:34.536743 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.536591 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:10:34.536743 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.536594 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:10:34.536743 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.536564 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/78ce6eba-2e3e-4a0e-96ad-13be205f151e-crio-socket\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.536743 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.536632 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls podName:78ce6eba-2e3e-4a0e-96ad-13be205f151e nodeName:}" failed. No retries permitted until 2026-09-09 21:10:35.036619833 +0000 UTC m=+37.549356629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jvgd8" (UID: "78ce6eba-2e3e-4a0e-96ad-13be205f151e") : secret "insights-runtime-extractor-tls" not found Sep 09 21:10:34.536743 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:34.536650 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls podName:d73b7970-9da3-44fb-bd20-efcc68505662 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:35.536639626 +0000 UTC m=+38.049376423 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls") pod "dns-default-jcd2g" (UID: "d73b7970-9da3-44fb-bd20-efcc68505662") : secret "dns-default-metrics-tls" not found Sep 09 21:10:34.536978 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.536767 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/78ce6eba-2e3e-4a0e-96ad-13be205f151e-data-volume\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:34.551328 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:34.551284 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmbpv\" (UniqueName: \"kubernetes.io/projected/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-api-access-bmbpv\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:35.041081 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.041039 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:35.041691 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.041099 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:35.041691 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:35.041172 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:10:35.041691 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:35.041200 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:10:35.041691 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:35.041224 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm podName:78ce6eba-2e3e-4a0e-96ad-13be205f151e nodeName:}" failed. No retries permitted until 2026-09-09 21:10:36.041211608 +0000 UTC m=+38.553948405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jvgd8" (UID: "78ce6eba-2e3e-4a0e-96ad-13be205f151e") : configmap "kube-rbac-proxy" not found Sep 09 21:10:35.041691 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:35.041252 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls podName:78ce6eba-2e3e-4a0e-96ad-13be205f151e nodeName:}" failed. No retries permitted until 2026-09-09 21:10:36.041239285 +0000 UTC m=+38.553976086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jvgd8" (UID: "78ce6eba-2e3e-4a0e-96ad-13be205f151e") : secret "insights-runtime-extractor-tls" not found Sep 09 21:10:35.044136 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.044112 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4"] Sep 09 21:10:35.045967 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.045954 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4" Sep 09 21:10:35.048256 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.048233 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 09 21:10:35.048388 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.048240 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 09 21:10:35.048445 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.048413 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5fkjf\"" Sep 09 21:10:35.059806 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.059783 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4"] Sep 09 21:10:35.142336 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.142283 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqzwx\" (UniqueName: \"kubernetes.io/projected/23f7aac5-fb7d-465c-a414-20b2b4689e25-kube-api-access-qqzwx\") pod \"migrator-fdbb8fdff-qvnh4\" (UID: \"23f7aac5-fb7d-465c-a414-20b2b4689e25\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4" Sep 09 21:10:35.152921 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.152890 2550 generic.go:356] "Generic (PLEG): container finished" podID="0a04424e-a63f-45bc-bd8b-53b3281e0982" containerID="505db06c051ff5c53003bbd1aa4dc75108beb428a6578b627091353ce00abcba" exitCode=0 Sep 09 21:10:35.153063 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.152960 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vth7c" event={"ID":"0a04424e-a63f-45bc-bd8b-53b3281e0982","Type":"ContainerDied","Data":"505db06c051ff5c53003bbd1aa4dc75108beb428a6578b627091353ce00abcba"} Sep 09 21:10:35.154176 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.154156 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jq8z4" event={"ID":"3ac31950-bf71-41ae-97ce-a45f4533baf0","Type":"ContainerStarted","Data":"eeffabcf852b311fa68e65673bfe2f30718b769dab68ae5cecb0f01ff98590ee"} Sep 09 21:10:35.243557 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.243531 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqzwx\" (UniqueName: \"kubernetes.io/projected/23f7aac5-fb7d-465c-a414-20b2b4689e25-kube-api-access-qqzwx\") pod \"migrator-fdbb8fdff-qvnh4\" (UID: \"23f7aac5-fb7d-465c-a414-20b2b4689e25\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4" Sep 09 21:10:35.253634 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.253571 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqzwx\" (UniqueName: \"kubernetes.io/projected/23f7aac5-fb7d-465c-a414-20b2b4689e25-kube-api-access-qqzwx\") pod \"migrator-fdbb8fdff-qvnh4\" (UID: \"23f7aac5-fb7d-465c-a414-20b2b4689e25\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4" Sep 09 21:10:35.354580 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.354511 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4" Sep 09 21:10:35.546839 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.546789 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:35.547024 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.546869 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:35.547024 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:35.546975 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:10:35.547024 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:35.546972 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:10:35.547175 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:35.547052 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls podName:d73b7970-9da3-44fb-bd20-efcc68505662 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:37.547030541 +0000 UTC m=+40.059767355 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls") pod "dns-default-jcd2g" (UID: "d73b7970-9da3-44fb-bd20-efcc68505662") : secret "dns-default-metrics-tls" not found Sep 09 21:10:35.547175 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:35.546986 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864846d867-ptrds: secret "image-registry-tls" not found Sep 09 21:10:35.547175 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:35.547123 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls podName:eca72d7b-7575-4cec-881d-d1d9ae20f872 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:37.54710609 +0000 UTC m=+40.059842886 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls") pod "image-registry-864846d867-ptrds" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872") : secret "image-registry-tls" not found Sep 09 21:10:35.636263 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.636224 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4"] Sep 09 21:10:35.641500 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:35.641466 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23f7aac5_fb7d_465c_a414_20b2b4689e25.slice/crio-ae4aef92100c91186c7cb3825b07997b0bbb07c4d27f13dcd2a38b9dfc5808ac WatchSource:0}: Error finding container ae4aef92100c91186c7cb3825b07997b0bbb07c4d27f13dcd2a38b9dfc5808ac: Status 404 returned error can't find the container with id ae4aef92100c91186c7cb3825b07997b0bbb07c4d27f13dcd2a38b9dfc5808ac Sep 09 21:10:35.682956 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:35.682769 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h8h2f_4ab41251-7b22-458e-83c1-4dc1ed95611a/dns-node-resolver/0.log" Sep 09 21:10:36.051164 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:36.051117 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:36.051652 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:36.051202 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:36.051652 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:36.051234 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:10:36.051652 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:36.051302 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls podName:78ce6eba-2e3e-4a0e-96ad-13be205f151e nodeName:}" failed. No retries permitted until 2026-09-09 21:10:38.051287169 +0000 UTC m=+40.564023984 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jvgd8" (UID: "78ce6eba-2e3e-4a0e-96ad-13be205f151e") : secret "insights-runtime-extractor-tls" not found Sep 09 21:10:36.051652 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:36.051368 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:10:36.051652 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:36.051435 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm podName:78ce6eba-2e3e-4a0e-96ad-13be205f151e nodeName:}" failed. No retries permitted until 2026-09-09 21:10:38.051418132 +0000 UTC m=+40.564154930 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jvgd8" (UID: "78ce6eba-2e3e-4a0e-96ad-13be205f151e") : configmap "kube-rbac-proxy" not found Sep 09 21:10:36.158698 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:36.158661 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4" event={"ID":"23f7aac5-fb7d-465c-a414-20b2b4689e25","Type":"ContainerStarted","Data":"ae4aef92100c91186c7cb3825b07997b0bbb07c4d27f13dcd2a38b9dfc5808ac"} Sep 09 21:10:36.166384 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:36.166336 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vth7c" event={"ID":"0a04424e-a63f-45bc-bd8b-53b3281e0982","Type":"ContainerStarted","Data":"3dada09ab259118e444fb52e03f6292433ad830ebe1f6145d11ccddec2159a76"} Sep 09 21:10:36.202986 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:36.202918 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vth7c" podStartSLOduration=9.110401203 podStartE2EDuration="38.202896016s" totalImagesPullingTime="29.092494813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:36.202199217 +0000 UTC m=+38.714936073" watchObservedRunningTime="2026-09-09 21:10:36.202896016 +0000 UTC m=+38.715632838" Sep 09 21:10:36.883384 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:36.883351 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rx56w_fdd3337d-7102-4a4b-826f-093466b1014d/node-ca/0.log" Sep 09 21:10:36.993702 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:36.993599 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-xcld4"] Sep 09 21:10:36.998384 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:36.998359 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-xcld4" Sep 09 21:10:37.000805 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.000774 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 09 21:10:37.000923 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.000860 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5g2ww\"" Sep 09 21:10:37.001569 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.001549 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 09 21:10:37.001660 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.001581 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 09 21:10:37.002197 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.002154 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 09 21:10:37.005828 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.005784 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-xcld4"] Sep 09 21:10:37.161112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.161021 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d06cd2d1-2b53-4123-bd8a-8248762c8650-signing-cabundle\") pod \"service-ca-75d99c484-xcld4\" (UID: \"d06cd2d1-2b53-4123-bd8a-8248762c8650\") " pod="openshift-service-ca/service-ca-75d99c484-xcld4" Sep 09 21:10:37.161112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.161069 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82pd2\" (UniqueName: \"kubernetes.io/projected/d06cd2d1-2b53-4123-bd8a-8248762c8650-kube-api-access-82pd2\") pod \"service-ca-75d99c484-xcld4\" (UID: \"d06cd2d1-2b53-4123-bd8a-8248762c8650\") " pod="openshift-service-ca/service-ca-75d99c484-xcld4" Sep 09 21:10:37.161656 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.161126 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d06cd2d1-2b53-4123-bd8a-8248762c8650-signing-key\") pod \"service-ca-75d99c484-xcld4\" (UID: \"d06cd2d1-2b53-4123-bd8a-8248762c8650\") " pod="openshift-service-ca/service-ca-75d99c484-xcld4" Sep 09 21:10:37.261964 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.261924 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d06cd2d1-2b53-4123-bd8a-8248762c8650-signing-cabundle\") pod \"service-ca-75d99c484-xcld4\" (UID: \"d06cd2d1-2b53-4123-bd8a-8248762c8650\") " pod="openshift-service-ca/service-ca-75d99c484-xcld4" Sep 09 21:10:37.261964 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.261961 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82pd2\" (UniqueName: \"kubernetes.io/projected/d06cd2d1-2b53-4123-bd8a-8248762c8650-kube-api-access-82pd2\") pod \"service-ca-75d99c484-xcld4\" (UID: \"d06cd2d1-2b53-4123-bd8a-8248762c8650\") " pod="openshift-service-ca/service-ca-75d99c484-xcld4" Sep 09 21:10:37.262182 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.262035 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d06cd2d1-2b53-4123-bd8a-8248762c8650-signing-key\") pod \"service-ca-75d99c484-xcld4\" (UID: \"d06cd2d1-2b53-4123-bd8a-8248762c8650\") " pod="openshift-service-ca/service-ca-75d99c484-xcld4" Sep 09 21:10:37.262888 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.262866 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d06cd2d1-2b53-4123-bd8a-8248762c8650-signing-cabundle\") pod \"service-ca-75d99c484-xcld4\" (UID: \"d06cd2d1-2b53-4123-bd8a-8248762c8650\") " pod="openshift-service-ca/service-ca-75d99c484-xcld4" Sep 09 21:10:37.265519 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.265495 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d06cd2d1-2b53-4123-bd8a-8248762c8650-signing-key\") pod \"service-ca-75d99c484-xcld4\" (UID: \"d06cd2d1-2b53-4123-bd8a-8248762c8650\") " pod="openshift-service-ca/service-ca-75d99c484-xcld4" Sep 09 21:10:37.270927 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.270902 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82pd2\" (UniqueName: \"kubernetes.io/projected/d06cd2d1-2b53-4123-bd8a-8248762c8650-kube-api-access-82pd2\") pod \"service-ca-75d99c484-xcld4\" (UID: \"d06cd2d1-2b53-4123-bd8a-8248762c8650\") " pod="openshift-service-ca/service-ca-75d99c484-xcld4" Sep 09 21:10:37.313500 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.313463 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-xcld4" Sep 09 21:10:37.564665 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.564586 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:37.564816 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.564684 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:37.564816 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:37.564737 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:10:37.564816 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:37.564758 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864846d867-ptrds: secret "image-registry-tls" not found Sep 09 21:10:37.564816 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:37.564802 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:10:37.564944 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:37.564837 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls podName:eca72d7b-7575-4cec-881d-d1d9ae20f872 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:41.56479914 +0000 UTC m=+44.077535937 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls") pod "image-registry-864846d867-ptrds" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872") : secret "image-registry-tls" not found Sep 09 21:10:37.564944 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:37.564851 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls podName:d73b7970-9da3-44fb-bd20-efcc68505662 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:41.564844502 +0000 UTC m=+44.077581300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls") pod "dns-default-jcd2g" (UID: "d73b7970-9da3-44fb-bd20-efcc68505662") : secret "dns-default-metrics-tls" not found Sep 09 21:10:37.869662 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:37.869632 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-xcld4"] Sep 09 21:10:37.872851 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:37.872827 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06cd2d1_2b53_4123_bd8a_8248762c8650.slice/crio-45d287027ba2da92c67290e244d032b5bee7f8aa3d800bb34bb40b7e6ecdc60b WatchSource:0}: Error finding container 45d287027ba2da92c67290e244d032b5bee7f8aa3d800bb34bb40b7e6ecdc60b: Status 404 returned error can't find the container with id 45d287027ba2da92c67290e244d032b5bee7f8aa3d800bb34bb40b7e6ecdc60b Sep 09 21:10:38.069335 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:38.069296 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:38.069504 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:38.069379 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:38.069504 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:38.069447 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:10:38.069632 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:38.069507 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:10:38.069632 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:38.069513 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls podName:78ce6eba-2e3e-4a0e-96ad-13be205f151e nodeName:}" failed. No retries permitted until 2026-09-09 21:10:42.069498249 +0000 UTC m=+44.582235047 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jvgd8" (UID: "78ce6eba-2e3e-4a0e-96ad-13be205f151e") : secret "insights-runtime-extractor-tls" not found Sep 09 21:10:38.069632 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:38.069577 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm podName:78ce6eba-2e3e-4a0e-96ad-13be205f151e nodeName:}" failed. No retries permitted until 2026-09-09 21:10:42.069557641 +0000 UTC m=+44.582294438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jvgd8" (UID: "78ce6eba-2e3e-4a0e-96ad-13be205f151e") : configmap "kube-rbac-proxy" not found Sep 09 21:10:38.172865 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:38.172828 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4" event={"ID":"23f7aac5-fb7d-465c-a414-20b2b4689e25","Type":"ContainerStarted","Data":"fe895236442dbd3a97f9bd217bc447338e61988b50e76721c6c44d39da80b99f"} Sep 09 21:10:38.173360 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:38.172898 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4" event={"ID":"23f7aac5-fb7d-465c-a414-20b2b4689e25","Type":"ContainerStarted","Data":"cc3ed6aaa7078c785a72f3e70853e7ff335d92216c27967f3e8e832760d7376a"} Sep 09 21:10:38.174382 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:38.174357 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jq8z4" event={"ID":"3ac31950-bf71-41ae-97ce-a45f4533baf0","Type":"ContainerStarted","Data":"82623e13bd90384b43b722bf82e4edfa5718c1a2d751e0e32a5f1b9505b3629d"} Sep 09 21:10:38.175501 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:38.175483 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-xcld4" event={"ID":"d06cd2d1-2b53-4123-bd8a-8248762c8650","Type":"ContainerStarted","Data":"45d287027ba2da92c67290e244d032b5bee7f8aa3d800bb34bb40b7e6ecdc60b"} Sep 09 21:10:38.190292 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:38.190217 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qvnh4" podStartSLOduration=1.235956071 podStartE2EDuration="3.190202532s" totalImagesPullingTime="1.954246461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:38.188828885 +0000 UTC m=+40.701565728" watchObservedRunningTime="2026-09-09 21:10:38.190202532 +0000 UTC m=+40.702939350" Sep 09 21:10:38.206176 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:38.206127 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jq8z4" podStartSLOduration=2.047237501 podStartE2EDuration="5.206110825s" totalImagesPullingTime="3.158873324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:38.205734048 +0000 UTC m=+40.718470867" watchObservedRunningTime="2026-09-09 21:10:38.206110825 +0000 UTC m=+40.718847638" Sep 09 21:10:40.183075 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:40.182990 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-xcld4" event={"ID":"d06cd2d1-2b53-4123-bd8a-8248762c8650","Type":"ContainerStarted","Data":"b5993e482671b2f9f6f704edd28fd3fce83614d26a3c28c8f7c79284901973ec"} Sep 09 21:10:40.200251 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:40.200191 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-xcld4" podStartSLOduration=2.167587997 podStartE2EDuration="4.200174046s" totalImagesPullingTime="2.032586049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:40.199226402 +0000 UTC m=+42.711963242" watchObservedRunningTime="2026-09-09 21:10:40.200174046 +0000 UTC m=+42.712910907" Sep 09 21:10:41.600044 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:41.600009 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:41.600512 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:41.600078 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:41.600512 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:41.600167 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:10:41.600512 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:41.600223 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls podName:d73b7970-9da3-44fb-bd20-efcc68505662 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:49.600209628 +0000 UTC m=+52.112946424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls") pod "dns-default-jcd2g" (UID: "d73b7970-9da3-44fb-bd20-efcc68505662") : secret "dns-default-metrics-tls" not found Sep 09 21:10:41.600512 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:41.600256 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:10:41.600512 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:41.600279 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864846d867-ptrds: secret "image-registry-tls" not found Sep 09 21:10:41.600512 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:41.600358 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls podName:eca72d7b-7575-4cec-881d-d1d9ae20f872 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:49.600337084 +0000 UTC m=+52.113073901 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls") pod "image-registry-864846d867-ptrds" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872") : secret "image-registry-tls" not found Sep 09 21:10:42.105110 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:42.105069 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:42.105422 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:42.105154 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:42.105422 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:42.105229 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:10:42.105422 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:10:42.105301 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls podName:78ce6eba-2e3e-4a0e-96ad-13be205f151e nodeName:}" failed. No retries permitted until 2026-09-09 21:10:50.105283279 +0000 UTC m=+52.618020079 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jvgd8" (UID: "78ce6eba-2e3e-4a0e-96ad-13be205f151e") : secret "insights-runtime-extractor-tls" not found Sep 09 21:10:42.105811 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:42.105790 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/78ce6eba-2e3e-4a0e-96ad-13be205f151e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:49.666954 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:49.666911 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:49.667363 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:49.666992 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:49.669557 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:49.669535 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d73b7970-9da3-44fb-bd20-efcc68505662-metrics-tls\") pod \"dns-default-jcd2g\" (UID: \"d73b7970-9da3-44fb-bd20-efcc68505662\") " pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:49.669619 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:49.669562 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls\") pod \"image-registry-864846d867-ptrds\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:49.718897 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:49.718856 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:49.734032 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:49.734007 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:50.019704 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:50.019611 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-864846d867-ptrds"] Sep 09 21:10:50.022823 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:50.022800 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca72d7b_7575_4cec_881d_d1d9ae20f872.slice/crio-19ffa46396440f63de69427e73e34af5f08da5b86f1153f42743459128004c01 WatchSource:0}: Error finding container 19ffa46396440f63de69427e73e34af5f08da5b86f1153f42743459128004c01: Status 404 returned error can't find the container with id 19ffa46396440f63de69427e73e34af5f08da5b86f1153f42743459128004c01 Sep 09 21:10:50.048051 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:50.048031 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jcd2g"] Sep 09 21:10:50.049734 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:50.049696 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd73b7970_9da3_44fb_bd20_efcc68505662.slice/crio-4bbe57c75f75f4b51070ec6aeeea9c129c963bc932e12ba3069136bccead70a3 WatchSource:0}: Error finding container 4bbe57c75f75f4b51070ec6aeeea9c129c963bc932e12ba3069136bccead70a3: Status 404 returned error can't find the container with id 4bbe57c75f75f4b51070ec6aeeea9c129c963bc932e12ba3069136bccead70a3 Sep 09 21:10:50.171373 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:50.171314 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:50.173788 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:50.173769 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/78ce6eba-2e3e-4a0e-96ad-13be205f151e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jvgd8\" (UID: \"78ce6eba-2e3e-4a0e-96ad-13be205f151e\") " pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:50.209651 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:50.209615 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerStarted","Data":"c3fe127d3a1f032b0cc43b4995d82efcd72ef4be7cf72c30740f492fc9a2b765"} Sep 09 21:10:50.209820 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:50.209663 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerStarted","Data":"19ffa46396440f63de69427e73e34af5f08da5b86f1153f42743459128004c01"} Sep 09 21:10:50.209820 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:50.209694 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:10:50.210775 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:50.210755 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcd2g" event={"ID":"d73b7970-9da3-44fb-bd20-efcc68505662","Type":"ContainerStarted","Data":"4bbe57c75f75f4b51070ec6aeeea9c129c963bc932e12ba3069136bccead70a3"} Sep 09 21:10:50.230232 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:50.230182 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-864846d867-ptrds" podStartSLOduration=30.23016699 podStartE2EDuration="30.23016699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:50.229560317 +0000 UTC m=+52.742297173" watchObservedRunningTime="2026-09-09 21:10:50.23016699 +0000 UTC m=+52.742903808" Sep 09 21:10:50.234984 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:50.234963 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jvgd8" Sep 09 21:10:50.518866 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:50.518832 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jvgd8"] Sep 09 21:10:50.522961 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:10:50.522920 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ce6eba_2e3e_4a0e_96ad_13be205f151e.slice/crio-8f3b3f07667d43256f62fd76a14a7dfec7451fad028eb7a99823cba6a085c093 WatchSource:0}: Error finding container 8f3b3f07667d43256f62fd76a14a7dfec7451fad028eb7a99823cba6a085c093: Status 404 returned error can't find the container with id 8f3b3f07667d43256f62fd76a14a7dfec7451fad028eb7a99823cba6a085c093 Sep 09 21:10:51.216121 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:51.216075 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jvgd8" event={"ID":"78ce6eba-2e3e-4a0e-96ad-13be205f151e","Type":"ContainerStarted","Data":"6f835b51ee1871231f2df01def88891b83994d756e3cf2578b131257b862382e"} Sep 09 21:10:51.216654 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:51.216161 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jvgd8" event={"ID":"78ce6eba-2e3e-4a0e-96ad-13be205f151e","Type":"ContainerStarted","Data":"8f3b3f07667d43256f62fd76a14a7dfec7451fad028eb7a99823cba6a085c093"} Sep 09 21:10:52.222225 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:52.222180 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcd2g" event={"ID":"d73b7970-9da3-44fb-bd20-efcc68505662","Type":"ContainerStarted","Data":"6bb2e5c8338e6df6ebf93a73961f34dbf2206d49528382dc6169b3525ff8f2e3"} Sep 09 21:10:52.222750 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:52.222251 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcd2g" event={"ID":"d73b7970-9da3-44fb-bd20-efcc68505662","Type":"ContainerStarted","Data":"4039f539b74221455d36801e84a4b1084d1c150807dfd285586a12827deace13"} Sep 09 21:10:52.222750 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:52.222334 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jcd2g" Sep 09 21:10:52.224654 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:52.224633 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jvgd8" event={"ID":"78ce6eba-2e3e-4a0e-96ad-13be205f151e","Type":"ContainerStarted","Data":"0464a586878ca5da242fa0cd80c8d41edc33f7bb229794d1edb55afa070bbfa5"} Sep 09 21:10:52.242085 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:52.242031 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jcd2g" podStartSLOduration=17.684944462 podStartE2EDuration="19.242014239s" totalImagesPullingTime="1.557069777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:52.241492371 +0000 UTC m=+54.754229213" watchObservedRunningTime="2026-09-09 21:10:52.242014239 +0000 UTC m=+54.754751058" Sep 09 21:10:53.230862 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:53.230763 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jvgd8" event={"ID":"78ce6eba-2e3e-4a0e-96ad-13be205f151e","Type":"ContainerStarted","Data":"4c533afb522137e1147a48f70097c2815786fb034283c3c3f580678f738a2505"} Sep 09 21:10:53.249592 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:53.249527 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jvgd8" podStartSLOduration=17.079622392 podStartE2EDuration="19.249503055s" totalImagesPullingTime="2.169880663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:10:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:53.2487447 +0000 UTC m=+55.761481530" watchObservedRunningTime="2026-09-09 21:10:53.249503055 +0000 UTC m=+55.762239876" Sep 09 21:10:57.143513 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:10:57.143483 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fr2g6" Sep 09 21:11:02.233246 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:02.233211 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jcd2g" Sep 09 21:11:03.769705 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:03.769659 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:11:03.770152 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:03.769729 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:11:03.772505 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:03.772480 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 21:11:03.773203 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:03.773185 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 21:11:03.782279 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:03.782240 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b343c4d-2b12-4093-b6de-c4223e67f2bf-metrics-certs\") pod \"network-metrics-daemon-9jn8c\" (UID: \"4b343c4d-2b12-4093-b6de-c4223e67f2bf\") " pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:11:03.782536 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:03.782521 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4211bff4-82e4-4201-b854-de56fd0b7e14-original-pull-secret\") pod \"global-pull-secret-syncer-kggbk\" (UID: \"4211bff4-82e4-4201-b854-de56fd0b7e14\") " pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:11:03.870643 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:03.870607 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb97w\" (UniqueName: \"kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w\") pod \"network-check-target-xctpc\" (UID: \"537bb9e7-2c10-4fcf-a29c-352599686ffa\") " pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:11:03.873635 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:03.873612 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 21:11:03.884127 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:03.884099 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 21:11:03.894597 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:03.894574 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb97w\" (UniqueName: \"kubernetes.io/projected/537bb9e7-2c10-4fcf-a29c-352599686ffa-kube-api-access-pb97w\") pod \"network-check-target-xctpc\" (UID: \"537bb9e7-2c10-4fcf-a29c-352599686ffa\") " pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:11:03.997396 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:03.997356 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7j9l\"" Sep 09 21:11:04.000454 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:04.000433 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kggbk" Sep 09 21:11:04.004687 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:04.004668 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:11:04.012334 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:04.009582 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ck66v\"" Sep 09 21:11:04.014751 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:04.014734 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jn8c" Sep 09 21:11:04.298153 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:04.298074 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kggbk"] Sep 09 21:11:04.301259 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:11:04.301234 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4211bff4_82e4_4201_b854_de56fd0b7e14.slice/crio-6258c79731743e66513105cf9fc33357c913b228161c5ef91bcc432fe9568acc WatchSource:0}: Error finding container 6258c79731743e66513105cf9fc33357c913b228161c5ef91bcc432fe9568acc: Status 404 returned error can't find the container with id 6258c79731743e66513105cf9fc33357c913b228161c5ef91bcc432fe9568acc Sep 09 21:11:04.328699 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:04.328594 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xctpc"] Sep 09 21:11:04.331288 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:11:04.331258 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod537bb9e7_2c10_4fcf_a29c_352599686ffa.slice/crio-b7207c08875de423be85461ebc28db6581119b036639b3014be8e1cd340103c2 WatchSource:0}: Error finding container b7207c08875de423be85461ebc28db6581119b036639b3014be8e1cd340103c2: Status 404 returned error can't find the container with id b7207c08875de423be85461ebc28db6581119b036639b3014be8e1cd340103c2 Sep 09 21:11:04.358623 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:04.358593 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9jn8c"] Sep 09 21:11:04.361660 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:11:04.361637 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b343c4d_2b12_4093_b6de_c4223e67f2bf.slice/crio-6343d273787f4fdb143ca3e1d299f5b27a917cffbaeb65d40882ee4430304743 WatchSource:0}: Error finding container 6343d273787f4fdb143ca3e1d299f5b27a917cffbaeb65d40882ee4430304743: Status 404 returned error can't find the container with id 6343d273787f4fdb143ca3e1d299f5b27a917cffbaeb65d40882ee4430304743 Sep 09 21:11:05.270646 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:05.270603 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kggbk" event={"ID":"4211bff4-82e4-4201-b854-de56fd0b7e14","Type":"ContainerStarted","Data":"6258c79731743e66513105cf9fc33357c913b228161c5ef91bcc432fe9568acc"} Sep 09 21:11:05.272418 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:05.272281 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jn8c" event={"ID":"4b343c4d-2b12-4093-b6de-c4223e67f2bf","Type":"ContainerStarted","Data":"6343d273787f4fdb143ca3e1d299f5b27a917cffbaeb65d40882ee4430304743"} Sep 09 21:11:05.273891 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:05.273866 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xctpc" event={"ID":"537bb9e7-2c10-4fcf-a29c-352599686ffa","Type":"ContainerStarted","Data":"b7207c08875de423be85461ebc28db6581119b036639b3014be8e1cd340103c2"} Sep 09 21:11:06.280613 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:06.280507 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jn8c" event={"ID":"4b343c4d-2b12-4093-b6de-c4223e67f2bf","Type":"ContainerStarted","Data":"8250e388e646e434e5ee62108e5e241d02e663a35fade1afd8750225d403b598"} Sep 09 21:11:06.280613 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:06.280574 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jn8c" event={"ID":"4b343c4d-2b12-4093-b6de-c4223e67f2bf","Type":"ContainerStarted","Data":"ba7c3d2bf36ec35277c9c4dcee883fb7d8b477b0e0133d92dbfb13bf2caf56ee"} Sep 09 21:11:06.300272 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:06.300211 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9jn8c" podStartSLOduration=66.974452193 podStartE2EDuration="1m8.30018844s" totalImagesPullingTime="1.325736247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:06.298451918 +0000 UTC m=+68.811188757" watchObservedRunningTime="2026-09-09 21:11:06.30018844 +0000 UTC m=+68.812925261" Sep 09 21:11:09.295137 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:09.295107 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xctpc" event={"ID":"537bb9e7-2c10-4fcf-a29c-352599686ffa","Type":"ContainerStarted","Data":"e7dfe3930f7487d851086347d512a688550d6f75789184051f0d6a9cecc478e7"} Sep 09 21:11:09.295531 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:09.295247 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:11:09.312057 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:09.311999 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xctpc" podStartSLOduration=66.45380832 podStartE2EDuration="1m11.311982938s" totalImagesPullingTime="4.858174618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:09.311459182 +0000 UTC m=+71.824196014" watchObservedRunningTime="2026-09-09 21:11:09.311982938 +0000 UTC m=+71.824719757" Sep 09 21:11:09.723652 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:09.723619 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:09.723841 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:09.723677 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:10.300153 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:10.300117 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kggbk" event={"ID":"4211bff4-82e4-4201-b854-de56fd0b7e14","Type":"ContainerStarted","Data":"5b22cc2eb116a33dbfdd69f12aeb41e149808af66c24f83ccbfdae75874fc49c"} Sep 09 21:11:10.317505 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:10.317446 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kggbk" podStartSLOduration=67.424939247 podStartE2EDuration="1m12.317428616s" totalImagesPullingTime="4.892489369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:10.316345844 +0000 UTC m=+72.829082677" watchObservedRunningTime="2026-09-09 21:11:10.317428616 +0000 UTC m=+72.830165434" Sep 09 21:11:11.221145 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:11.221109 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:11.221350 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:11.221169 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:19.723658 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:19.723544 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:19.723658 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:19.723597 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:21.220101 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:21.220068 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:21.220505 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:21.220120 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:29.723257 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:29.723206 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:29.723673 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:29.723280 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:29.723673 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:29.723347 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:11:29.723809 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:29.723780 2550 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"c3fe127d3a1f032b0cc43b4995d82efcd72ef4be7cf72c30740f492fc9a2b765"} pod="openshift-image-registry/image-registry-864846d867-ptrds" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:11:29.727415 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:29.727385 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:29.727541 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:29.727429 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:39.727613 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:39.727579 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:39.728012 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:39.727638 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:40.302434 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:40.302398 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xctpc" Sep 09 21:11:49.728068 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:49.728034 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:49.728549 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:49.728096 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:54.742968 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:54.742921 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" containerID="cri-o://c3fe127d3a1f032b0cc43b4995d82efcd72ef4be7cf72c30740f492fc9a2b765" gracePeriod=30 Sep 09 21:11:55.440995 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:55.440954 2550 generic.go:356] "Generic (PLEG): container finished" podID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerID="c3fe127d3a1f032b0cc43b4995d82efcd72ef4be7cf72c30740f492fc9a2b765" exitCode=0 Sep 09 21:11:55.441274 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:55.441049 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerDied","Data":"c3fe127d3a1f032b0cc43b4995d82efcd72ef4be7cf72c30740f492fc9a2b765"} Sep 09 21:11:55.441274 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:55.441104 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerStarted","Data":"1fd9e0dd378a1e67060ae1d218b6e223070d82166160467eef7eb380b8c46b46"} Sep 09 21:11:55.441274 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:11:55.441132 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:12:09.723945 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:09.723913 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:09.724423 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:09.723963 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:16.448017 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:16.447982 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:16.448494 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:16.448051 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:19.722874 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:19.722834 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:19.723283 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:19.722898 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:26.448177 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:26.448143 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:26.448763 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:26.448203 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:29.723151 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:29.723116 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:29.723554 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:29.723179 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:29.723554 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:29.723218 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:12:29.723760 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:29.723741 2550 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"1fd9e0dd378a1e67060ae1d218b6e223070d82166160467eef7eb380b8c46b46"} pod="openshift-image-registry/image-registry-864846d867-ptrds" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:12:29.727197 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:29.727170 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:29.727306 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:29.727219 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:39.728228 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:39.728193 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:39.728661 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:39.728255 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:49.728083 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:49.728002 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:49.728083 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:49.728058 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:54.742781 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:54.742736 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" containerID="cri-o://1fd9e0dd378a1e67060ae1d218b6e223070d82166160467eef7eb380b8c46b46" gracePeriod=30 Sep 09 21:12:55.624424 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:55.624389 2550 generic.go:356] "Generic (PLEG): container finished" podID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerID="1fd9e0dd378a1e67060ae1d218b6e223070d82166160467eef7eb380b8c46b46" exitCode=0 Sep 09 21:12:55.624424 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:55.624426 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerDied","Data":"1fd9e0dd378a1e67060ae1d218b6e223070d82166160467eef7eb380b8c46b46"} Sep 09 21:12:55.624732 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:55.624467 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerStarted","Data":"42f17b2b5d19064c61a9c27168db7739b52154f97f71b88bb4f4fb3c6caa355f"} Sep 09 21:12:55.624732 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:55.624493 2550 scope.go:117] "RemoveContainer" containerID="c3fe127d3a1f032b0cc43b4995d82efcd72ef4be7cf72c30740f492fc9a2b765" Sep 09 21:12:55.624732 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:12:55.624623 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:13:09.723636 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:09.723600 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:09.724102 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:09.723694 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:16.633721 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:16.633682 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:16.634166 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:16.633752 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:19.723705 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:19.723664 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:19.724078 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:19.723730 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:26.633840 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:26.633798 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:26.634390 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:26.633855 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:29.723160 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:29.723126 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:29.723662 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:29.723185 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:29.723662 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:29.723223 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:13:29.723761 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:29.723674 2550 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"42f17b2b5d19064c61a9c27168db7739b52154f97f71b88bb4f4fb3c6caa355f"} pod="openshift-image-registry/image-registry-864846d867-ptrds" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:13:29.727117 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:29.727092 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:29.727244 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:29.727145 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:39.728142 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:39.728109 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:39.728567 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:39.728170 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:49.727890 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:49.727858 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:49.728260 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:49.727934 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:54.742778 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:54.742730 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" containerID="cri-o://42f17b2b5d19064c61a9c27168db7739b52154f97f71b88bb4f4fb3c6caa355f" gracePeriod=30 Sep 09 21:13:55.815580 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:55.815550 2550 generic.go:356] "Generic (PLEG): container finished" podID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerID="42f17b2b5d19064c61a9c27168db7739b52154f97f71b88bb4f4fb3c6caa355f" exitCode=0 Sep 09 21:13:55.816046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:55.815613 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerDied","Data":"42f17b2b5d19064c61a9c27168db7739b52154f97f71b88bb4f4fb3c6caa355f"} Sep 09 21:13:55.816046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:55.815653 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerStarted","Data":"bd03cfdde8782d58c276e96f67ae20bb6e6e8a600153be602b9bfb41af25aa8c"} Sep 09 21:13:55.816046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:55.815673 2550 scope.go:117] "RemoveContainer" containerID="1fd9e0dd378a1e67060ae1d218b6e223070d82166160467eef7eb380b8c46b46" Sep 09 21:13:55.816046 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:13:55.815850 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:14:09.724682 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:09.724647 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:09.725141 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:09.724708 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:16.825309 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:16.825272 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:16.825729 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:16.825343 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:19.723607 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:19.723526 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:19.723607 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:19.723580 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:26.825025 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:26.824986 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:26.825512 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:26.825050 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:29.723454 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:29.723421 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:29.723895 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:29.723484 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:29.723895 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:29.723525 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:14:29.724012 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:29.723949 2550 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"bd03cfdde8782d58c276e96f67ae20bb6e6e8a600153be602b9bfb41af25aa8c"} pod="openshift-image-registry/image-registry-864846d867-ptrds" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:14:29.727739 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:29.727709 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:29.727868 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:29.727761 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:39.728361 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:39.728307 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:39.728729 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:39.728388 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:49.727940 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:49.727906 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:49.728440 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:49.727982 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:54.742906 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:54.742859 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" containerID="cri-o://bd03cfdde8782d58c276e96f67ae20bb6e6e8a600153be602b9bfb41af25aa8c" gracePeriod=30 Sep 09 21:14:54.995427 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:54.995348 2550 generic.go:356] "Generic (PLEG): container finished" podID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerID="bd03cfdde8782d58c276e96f67ae20bb6e6e8a600153be602b9bfb41af25aa8c" exitCode=0 Sep 09 21:14:54.995427 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:54.995385 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerDied","Data":"bd03cfdde8782d58c276e96f67ae20bb6e6e8a600153be602b9bfb41af25aa8c"} Sep 09 21:14:54.995427 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:54.995425 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerStarted","Data":"a339107471134caf5eabe32a6ec95a64bdef2f8f5db946f029115c4ff2c1c287"} Sep 09 21:14:54.995687 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:54.995451 2550 scope.go:117] "RemoveContainer" containerID="42f17b2b5d19064c61a9c27168db7739b52154f97f71b88bb4f4fb3c6caa355f" Sep 09 21:14:54.995687 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:54.995590 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:14:57.851551 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:14:57.851526 2550 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 21:15:09.723340 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:09.723273 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:09.726633 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:09.723354 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:16.004932 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:16.004888 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:16.005454 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:16.004955 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:19.723948 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:19.723913 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:19.724424 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:19.723984 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:26.004226 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:26.004187 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:26.004723 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:26.004241 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:29.725967 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:29.725934 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:29.726386 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:29.725986 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:29.726386 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:29.726030 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:15:29.726511 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:29.726489 2550 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"a339107471134caf5eabe32a6ec95a64bdef2f8f5db946f029115c4ff2c1c287"} pod="openshift-image-registry/image-registry-864846d867-ptrds" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:15:29.730270 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:29.730242 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:29.730419 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:29.730289 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:39.730541 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:39.730503 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:39.730905 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:39.730564 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:49.730554 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:49.730467 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:49.730554 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:49.730515 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:54.745747 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:54.745711 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" containerID="cri-o://a339107471134caf5eabe32a6ec95a64bdef2f8f5db946f029115c4ff2c1c287" gracePeriod=30 Sep 09 21:15:55.179399 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:55.179367 2550 generic.go:356] "Generic (PLEG): container finished" podID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerID="a339107471134caf5eabe32a6ec95a64bdef2f8f5db946f029115c4ff2c1c287" exitCode=0 Sep 09 21:15:55.179557 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:55.179405 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerDied","Data":"a339107471134caf5eabe32a6ec95a64bdef2f8f5db946f029115c4ff2c1c287"} Sep 09 21:15:55.179557 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:55.179453 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerStarted","Data":"c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066"} Sep 09 21:15:55.179557 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:55.179478 2550 scope.go:117] "RemoveContainer" containerID="bd03cfdde8782d58c276e96f67ae20bb6e6e8a600153be602b9bfb41af25aa8c" Sep 09 21:15:55.179703 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:15:55.179615 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:16:09.723455 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:09.723422 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:09.723994 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:09.723479 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:16.189172 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:16.189134 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:16.189642 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:16.189188 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:19.723801 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:19.723766 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:19.724178 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:19.723819 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:26.188799 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:26.188762 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:26.189248 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:26.188817 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:29.723570 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:29.723538 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:29.723946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:29.723596 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:29.723946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:29.723633 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:16:29.724080 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:29.724056 2550 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066"} pod="openshift-image-registry/image-registry-864846d867-ptrds" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:16:29.727674 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:29.727652 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:29.727798 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:29.727691 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:39.728543 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:39.728503 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:39.728943 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:39.728575 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:49.728399 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:49.728366 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:49.728837 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:49.728421 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:54.742938 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:54.742891 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" containerID="cri-o://c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066" gracePeriod=30 Sep 09 21:16:54.859883 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:16:54.859856 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-864846d867-ptrds_openshift-image-registry(eca72d7b-7575-4cec-881d-d1d9ae20f872)\"" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" Sep 09 21:16:55.364309 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:55.364278 2550 generic.go:356] "Generic (PLEG): container finished" podID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerID="c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066" exitCode=0 Sep 09 21:16:55.364501 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:55.364351 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerDied","Data":"c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066"} Sep 09 21:16:55.364501 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:55.364409 2550 scope.go:117] "RemoveContainer" containerID="a339107471134caf5eabe32a6ec95a64bdef2f8f5db946f029115c4ff2c1c287" Sep 09 21:16:55.364788 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:16:55.364774 2550 scope.go:117] "RemoveContainer" containerID="c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066" Sep 09 21:16:55.364994 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:16:55.364971 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-864846d867-ptrds_openshift-image-registry(eca72d7b-7575-4cec-881d-d1d9ae20f872)\"" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" Sep 09 21:17:07.986257 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:17:07.986224 2550 scope.go:117] "RemoveContainer" containerID="c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066" Sep 09 21:17:07.986692 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:17:07.986416 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-864846d867-ptrds_openshift-image-registry(eca72d7b-7575-4cec-881d-d1d9ae20f872)\"" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" Sep 09 21:17:18.984673 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:17:18.984596 2550 scope.go:117] "RemoveContainer" containerID="c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066" Sep 09 21:17:18.985056 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:17:18.984756 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-864846d867-ptrds_openshift-image-registry(eca72d7b-7575-4cec-881d-d1d9ae20f872)\"" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" Sep 09 21:17:30.984620 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:17:30.984589 2550 scope.go:117] "RemoveContainer" containerID="c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066" Sep 09 21:17:30.985081 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:17:30.984755 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-864846d867-ptrds_openshift-image-registry(eca72d7b-7575-4cec-881d-d1d9ae20f872)\"" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" Sep 09 21:17:44.984772 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:17:44.984731 2550 scope.go:117] "RemoveContainer" containerID="c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066" Sep 09 21:17:44.985196 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:17:44.984926 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-864846d867-ptrds_openshift-image-registry(eca72d7b-7575-4cec-881d-d1d9ae20f872)\"" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" Sep 09 21:17:59.985209 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:17:59.985183 2550 scope.go:117] "RemoveContainer" containerID="c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066" Sep 09 21:17:59.985689 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:17:59.985351 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-864846d867-ptrds_openshift-image-registry(eca72d7b-7575-4cec-881d-d1d9ae20f872)\"" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" Sep 09 21:18:10.984687 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:10.984650 2550 scope.go:117] "RemoveContainer" containerID="c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066" Sep 09 21:18:10.985168 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:18:10.984806 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-864846d867-ptrds_openshift-image-registry(eca72d7b-7575-4cec-881d-d1d9ae20f872)\"" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" Sep 09 21:18:22.985631 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:22.985595 2550 scope.go:117] "RemoveContainer" containerID="c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066" Sep 09 21:18:23.627190 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:23.627153 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerStarted","Data":"b9c1089b7819058b31e728a2514649abe434b43f84b24f7357a9719174be9751"} Sep 09 21:18:23.627384 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:23.627369 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:18:39.723780 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:39.723742 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:18:39.724249 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:39.723807 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:18:44.636545 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:44.636510 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:18:44.636967 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:44.636563 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:18:49.723311 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:49.723228 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:18:49.723311 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:49.723290 2550 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:18:53.547643 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.547615 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx"] Sep 09 21:18:53.550733 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.550708 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7"] Sep 09 21:18:53.550877 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.550828 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx" Sep 09 21:18:53.554092 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.554053 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.554674 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.554658 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 09 21:18:53.555041 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.555024 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 09 21:18:53.555109 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.555062 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 09 21:18:53.555745 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.555731 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 09 21:18:53.556790 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.556767 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 09 21:18:53.556893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.556767 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 09 21:18:53.556893 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.556837 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 09 21:18:53.557020 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.556902 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 09 21:18:53.558074 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.558053 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-g7gst\"" Sep 09 21:18:53.571786 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.571761 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx"] Sep 09 21:18:53.572601 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.572572 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7"] Sep 09 21:18:53.607493 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.607463 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f451aed-6ace-45aa-9fdc-8bc311e36d1a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-87df4c844-wk5tx\" (UID: \"8f451aed-6ace-45aa-9fdc-8bc311e36d1a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx" Sep 09 21:18:53.607624 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.607513 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.607624 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.607543 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.607624 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.607598 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tccc8\" (UniqueName: \"kubernetes.io/projected/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-kube-api-access-tccc8\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.607624 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.607616 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-ca\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.607806 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.607636 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.607806 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.607665 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmpqw\" (UniqueName: \"kubernetes.io/projected/8f451aed-6ace-45aa-9fdc-8bc311e36d1a-kube-api-access-tmpqw\") pod \"managed-serviceaccount-addon-agent-87df4c844-wk5tx\" (UID: \"8f451aed-6ace-45aa-9fdc-8bc311e36d1a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx" Sep 09 21:18:53.607806 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.607718 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-hub\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.608477 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.608460 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-864846d867-ptrds"] Sep 09 21:18:53.612503 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.612475 2550 patch_prober.go:28] interesting pod/image-registry-864846d867-ptrds container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:18:53.612595 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.612526 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:18:53.699415 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.699387 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d965ffd6c-xsjkg"] Sep 09 21:18:53.702557 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.702518 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.708766 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.708746 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39b28238-b7ad-42ea-ac40-8fd800000b8e-registry-certificates\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.708853 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.708776 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tccc8\" (UniqueName: \"kubernetes.io/projected/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-kube-api-access-tccc8\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.708853 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.708794 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-ca\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.708853 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.708811 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.708853 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.708834 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39b28238-b7ad-42ea-ac40-8fd800000b8e-bound-sa-token\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.709044 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.708864 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39b28238-b7ad-42ea-ac40-8fd800000b8e-image-registry-private-configuration\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.709044 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.708892 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmpqw\" (UniqueName: \"kubernetes.io/projected/8f451aed-6ace-45aa-9fdc-8bc311e36d1a-kube-api-access-tmpqw\") pod \"managed-serviceaccount-addon-agent-87df4c844-wk5tx\" (UID: \"8f451aed-6ace-45aa-9fdc-8bc311e36d1a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx" Sep 09 21:18:53.709044 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.708917 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djx4w\" (UniqueName: \"kubernetes.io/projected/39b28238-b7ad-42ea-ac40-8fd800000b8e-kube-api-access-djx4w\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.709044 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.708948 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-hub\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.709044 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.709008 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f451aed-6ace-45aa-9fdc-8bc311e36d1a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-87df4c844-wk5tx\" (UID: \"8f451aed-6ace-45aa-9fdc-8bc311e36d1a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx" Sep 09 21:18:53.709352 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.709046 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.709352 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.709077 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39b28238-b7ad-42ea-ac40-8fd800000b8e-ca-trust-extracted\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.709352 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.709104 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39b28238-b7ad-42ea-ac40-8fd800000b8e-registry-tls\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.709352 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.709139 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.709352 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.709183 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39b28238-b7ad-42ea-ac40-8fd800000b8e-trusted-ca\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.709352 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.709228 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39b28238-b7ad-42ea-ac40-8fd800000b8e-installation-pull-secrets\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.710010 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.709986 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.711452 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.711432 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.711899 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.711875 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-ca\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.711985 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.711953 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-hub\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.712354 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.712314 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.712404 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.712382 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f451aed-6ace-45aa-9fdc-8bc311e36d1a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-87df4c844-wk5tx\" (UID: \"8f451aed-6ace-45aa-9fdc-8bc311e36d1a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx" Sep 09 21:18:53.720091 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.720071 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d965ffd6c-xsjkg"] Sep 09 21:18:53.729224 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.729204 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tccc8\" (UniqueName: \"kubernetes.io/projected/84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702-kube-api-access-tccc8\") pod \"cluster-proxy-proxy-agent-cdb77955b-sdwf7\" (UID: \"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:53.729311 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.729294 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmpqw\" (UniqueName: \"kubernetes.io/projected/8f451aed-6ace-45aa-9fdc-8bc311e36d1a-kube-api-access-tmpqw\") pod \"managed-serviceaccount-addon-agent-87df4c844-wk5tx\" (UID: \"8f451aed-6ace-45aa-9fdc-8bc311e36d1a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx" Sep 09 21:18:53.810202 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.810125 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39b28238-b7ad-42ea-ac40-8fd800000b8e-registry-certificates\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.810202 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.810160 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39b28238-b7ad-42ea-ac40-8fd800000b8e-bound-sa-token\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.810202 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.810179 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39b28238-b7ad-42ea-ac40-8fd800000b8e-image-registry-private-configuration\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.810202 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.810195 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djx4w\" (UniqueName: \"kubernetes.io/projected/39b28238-b7ad-42ea-ac40-8fd800000b8e-kube-api-access-djx4w\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.810554 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.810227 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39b28238-b7ad-42ea-ac40-8fd800000b8e-ca-trust-extracted\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.810554 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.810250 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39b28238-b7ad-42ea-ac40-8fd800000b8e-registry-tls\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.810554 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.810281 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39b28238-b7ad-42ea-ac40-8fd800000b8e-trusted-ca\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.810554 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.810313 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39b28238-b7ad-42ea-ac40-8fd800000b8e-installation-pull-secrets\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.810733 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.810680 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39b28238-b7ad-42ea-ac40-8fd800000b8e-ca-trust-extracted\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.811193 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.811170 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39b28238-b7ad-42ea-ac40-8fd800000b8e-registry-certificates\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.811480 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.811452 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39b28238-b7ad-42ea-ac40-8fd800000b8e-trusted-ca\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.812806 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.812788 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39b28238-b7ad-42ea-ac40-8fd800000b8e-image-registry-private-configuration\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.812950 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.812921 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39b28238-b7ad-42ea-ac40-8fd800000b8e-installation-pull-secrets\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.813032 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.813017 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39b28238-b7ad-42ea-ac40-8fd800000b8e-registry-tls\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.821622 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.821596 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djx4w\" (UniqueName: \"kubernetes.io/projected/39b28238-b7ad-42ea-ac40-8fd800000b8e-kube-api-access-djx4w\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.822290 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.822269 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39b28238-b7ad-42ea-ac40-8fd800000b8e-bound-sa-token\") pod \"image-registry-d965ffd6c-xsjkg\" (UID: \"39b28238-b7ad-42ea-ac40-8fd800000b8e\") " pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:53.866795 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.866775 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx" Sep 09 21:18:53.873832 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:53.873815 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" Sep 09 21:18:54.039578 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:54.039550 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:54.139692 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:54.139669 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx"] Sep 09 21:18:54.142124 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:18:54.142099 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f451aed_6ace_45aa_9fdc_8bc311e36d1a.slice/crio-7377e7007639043166465acd8c4e3e35adef627be0cb19410defaac5ceaaa56a WatchSource:0}: Error finding container 7377e7007639043166465acd8c4e3e35adef627be0cb19410defaac5ceaaa56a: Status 404 returned error can't find the container with id 7377e7007639043166465acd8c4e3e35adef627be0cb19410defaac5ceaaa56a Sep 09 21:18:54.144515 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:54.144499 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:18:54.145099 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:54.145047 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7"] Sep 09 21:18:54.147153 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:18:54.147094 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84e2a8f1_4c3c_41d5_87a1_d5a5f0b2c702.slice/crio-e349fbdd65f7dbd88bd4011c9ba59c475824b9fc211b155f0a27a038c8874080 WatchSource:0}: Error finding container e349fbdd65f7dbd88bd4011c9ba59c475824b9fc211b155f0a27a038c8874080: Status 404 returned error can't find the container with id e349fbdd65f7dbd88bd4011c9ba59c475824b9fc211b155f0a27a038c8874080 Sep 09 21:18:54.310580 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:54.310490 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d965ffd6c-xsjkg"] Sep 09 21:18:54.312860 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:18:54.312836 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39b28238_b7ad_42ea_ac40_8fd800000b8e.slice/crio-84f24ac4e5efba58383ebb20f46bd1c8af319d84ea32290f3192c4187b16a77b WatchSource:0}: Error finding container 84f24ac4e5efba58383ebb20f46bd1c8af319d84ea32290f3192c4187b16a77b: Status 404 returned error can't find the container with id 84f24ac4e5efba58383ebb20f46bd1c8af319d84ea32290f3192c4187b16a77b Sep 09 21:18:54.724067 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:54.723969 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" event={"ID":"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702","Type":"ContainerStarted","Data":"e349fbdd65f7dbd88bd4011c9ba59c475824b9fc211b155f0a27a038c8874080"} Sep 09 21:18:54.728073 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:54.727115 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" event={"ID":"39b28238-b7ad-42ea-ac40-8fd800000b8e","Type":"ContainerStarted","Data":"50dc528172799bc6d7f96edc13d3774dab256c5d35699e68b552934129c04a73"} Sep 09 21:18:54.728073 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:54.727166 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" event={"ID":"39b28238-b7ad-42ea-ac40-8fd800000b8e","Type":"ContainerStarted","Data":"84f24ac4e5efba58383ebb20f46bd1c8af319d84ea32290f3192c4187b16a77b"} Sep 09 21:18:54.728073 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:54.728015 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:18:54.730313 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:54.730281 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx" event={"ID":"8f451aed-6ace-45aa-9fdc-8bc311e36d1a","Type":"ContainerStarted","Data":"7377e7007639043166465acd8c4e3e35adef627be0cb19410defaac5ceaaa56a"} Sep 09 21:18:54.752040 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:54.751984 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" podStartSLOduration=1.751962485 podStartE2EDuration="1.751962485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:18:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:18:54.749899372 +0000 UTC m=+537.262636205" watchObservedRunningTime="2026-09-09 21:18:54.751962485 +0000 UTC m=+537.264699306" Sep 09 21:18:58.748997 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:58.748960 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx" event={"ID":"8f451aed-6ace-45aa-9fdc-8bc311e36d1a","Type":"ContainerStarted","Data":"15e02adb85535a215b57bd91e09ec781d2ec3e7c89e744627c8225acd34ff552"} Sep 09 21:18:58.751067 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:58.751036 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" event={"ID":"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702","Type":"ContainerStarted","Data":"c089b80c417ee04e8b700e99c8aa9ffb4f5b46e62e01a8208089106e7fcd81e7"} Sep 09 21:18:58.751183 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:58.751078 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" event={"ID":"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702","Type":"ContainerStarted","Data":"7729d085d0cfa0d5a1781f625f3f3cc0fe4c46a94dfa5d6aa74c7cc15a445c95"} Sep 09 21:18:58.751183 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:58.751097 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" event={"ID":"84e2a8f1-4c3c-41d5-87a1-d5a5f0b2c702","Type":"ContainerStarted","Data":"1281f6280365b6491988888f30f9fd8aea1e109499058b5da94806c2953a81eb"} Sep 09 21:18:58.764782 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:58.764737 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-87df4c844-wk5tx" podStartSLOduration=1.9998671730000002 podStartE2EDuration="5.764720428s" totalImagesPullingTime="3.764853255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:18:58.764133989 +0000 UTC m=+541.276870821" watchObservedRunningTime="2026-09-09 21:18:58.764720428 +0000 UTC m=+541.277457248" Sep 09 21:18:58.782881 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:18:58.782839 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cdb77955b-sdwf7" podStartSLOduration=2.008131966 podStartE2EDuration="5.78282411s" totalImagesPullingTime="3.774692144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:18:58.782005783 +0000 UTC m=+541.294742613" watchObservedRunningTime="2026-09-09 21:18:58.78282411 +0000 UTC m=+541.295560929" Sep 09 21:19:03.614811 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.614734 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:19:03.622477 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.622452 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv"] Sep 09 21:19:03.627752 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.627737 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.632122 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.632100 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 09 21:19:03.632225 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.632120 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 09 21:19:03.641803 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.641782 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 21:19:03.642614 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.642593 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 21:19:03.642723 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.642600 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4gwcj\"" Sep 09 21:19:03.651407 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.651388 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gkwg5"] Sep 09 21:19:03.656121 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.656105 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv"] Sep 09 21:19:03.656380 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.656360 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.656380 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.656298 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 21:19:03.659400 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.659382 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 21:19:03.659755 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.659740 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bz8x8\"" Sep 09 21:19:03.660104 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.660080 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 21:19:03.661610 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.661592 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 21:19:03.683083 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683059 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.683197 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683115 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.683197 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683148 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a65e9c3-a61b-4ce5-baba-4b101f963a66-metrics-client-ca\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.683197 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683174 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-textfile\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.683344 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683203 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a65e9c3-a61b-4ce5-baba-4b101f963a66-root\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.683344 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683251 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szpv6\" (UniqueName: \"kubernetes.io/projected/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-kube-api-access-szpv6\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.683344 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683280 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-accelerators-collector-config\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.683344 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683307 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.683490 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683370 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.683490 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683401 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-tls\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.683490 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683428 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a65e9c3-a61b-4ce5-baba-4b101f963a66-sys\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.683490 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683451 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-wtmp\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.683490 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.683474 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlqst\" (UniqueName: \"kubernetes.io/projected/1a65e9c3-a61b-4ce5-baba-4b101f963a66-kube-api-access-hlqst\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.784598 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784562 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-tls\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.784796 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784603 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a65e9c3-a61b-4ce5-baba-4b101f963a66-sys\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.784796 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784628 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-wtmp\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.784796 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784654 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlqst\" (UniqueName: \"kubernetes.io/projected/1a65e9c3-a61b-4ce5-baba-4b101f963a66-kube-api-access-hlqst\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.784796 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784682 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.784796 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784726 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.784796 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:19:03.784728 2550 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 09 21:19:03.784796 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784749 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a65e9c3-a61b-4ce5-baba-4b101f963a66-metrics-client-ca\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.784796 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784773 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-textfile\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.784796 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784774 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-wtmp\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.784796 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:19:03.784792 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-tls podName:1a65e9c3-a61b-4ce5-baba-4b101f963a66 nodeName:}" failed. No retries permitted until 2026-09-09 21:19:04.284770931 +0000 UTC m=+546.797507733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-tls") pod "node-exporter-gkwg5" (UID: "1a65e9c3-a61b-4ce5-baba-4b101f963a66") : secret "node-exporter-tls" not found Sep 09 21:19:03.785361 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784832 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a65e9c3-a61b-4ce5-baba-4b101f963a66-root\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.785361 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784870 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szpv6\" (UniqueName: \"kubernetes.io/projected/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-kube-api-access-szpv6\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.785361 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784901 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-accelerators-collector-config\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.785361 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784931 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.785361 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.784982 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.785361 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.785082 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-textfile\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.785660 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.785452 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a65e9c3-a61b-4ce5-baba-4b101f963a66-sys\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.785660 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.785638 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a65e9c3-a61b-4ce5-baba-4b101f963a66-root\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.786224 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.786195 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-accelerators-collector-config\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.786372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.786351 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.786475 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.786457 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a65e9c3-a61b-4ce5-baba-4b101f963a66-metrics-client-ca\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.788248 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.788181 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.788603 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.788585 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.788864 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.788842 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.794165 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.794143 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlqst\" (UniqueName: \"kubernetes.io/projected/1a65e9c3-a61b-4ce5-baba-4b101f963a66-kube-api-access-hlqst\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:03.795767 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.795749 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szpv6\" (UniqueName: \"kubernetes.io/projected/ec6e14a4-77f8-4698-b2e7-7c6fada680d8-kube-api-access-szpv6\") pod \"openshift-state-metrics-657fd6c54-z4chv\" (UID: \"ec6e14a4-77f8-4698-b2e7-7c6fada680d8\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:03.937481 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:03.937456 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" Sep 09 21:19:04.218268 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.218197 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv"] Sep 09 21:19:04.222237 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:19:04.222208 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec6e14a4_77f8_4698_b2e7_7c6fada680d8.slice/crio-ac8769cf3f7ed34854dd43cac4db015c5ceae593159d3ef1476e021169119dd7 WatchSource:0}: Error finding container ac8769cf3f7ed34854dd43cac4db015c5ceae593159d3ef1476e021169119dd7: Status 404 returned error can't find the container with id ac8769cf3f7ed34854dd43cac4db015c5ceae593159d3ef1476e021169119dd7 Sep 09 21:19:04.288637 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.288612 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-tls\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:04.290876 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.290852 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a65e9c3-a61b-4ce5-baba-4b101f963a66-node-exporter-tls\") pod \"node-exporter-gkwg5\" (UID: \"1a65e9c3-a61b-4ce5-baba-4b101f963a66\") " pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:04.567955 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.567879 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gkwg5" Sep 09 21:19:04.578559 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:19:04.578538 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a65e9c3_a61b_4ce5_baba_4b101f963a66.slice/crio-4928734de90d1b80756e56b4a0a1c48d8908ecc642ed9607210729c7174d1641 WatchSource:0}: Error finding container 4928734de90d1b80756e56b4a0a1c48d8908ecc642ed9607210729c7174d1641: Status 404 returned error can't find the container with id 4928734de90d1b80756e56b4a0a1c48d8908ecc642ed9607210729c7174d1641 Sep 09 21:19:04.683712 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.683681 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 21:19:04.688524 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.688505 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.691846 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.691822 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 09 21:19:04.691949 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.691845 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-crhws\"" Sep 09 21:19:04.692098 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.692082 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 09 21:19:04.692146 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.692092 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 09 21:19:04.692146 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.692126 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 09 21:19:04.692146 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.692137 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 09 21:19:04.692382 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.692359 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 09 21:19:04.692382 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.692376 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 09 21:19:04.692545 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.692391 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 09 21:19:04.692600 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.692588 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 09 21:19:04.705892 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.705871 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 21:19:04.774424 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.774388 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gkwg5" event={"ID":"1a65e9c3-a61b-4ce5-baba-4b101f963a66","Type":"ContainerStarted","Data":"4928734de90d1b80756e56b4a0a1c48d8908ecc642ed9607210729c7174d1641"} Sep 09 21:19:04.776503 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.776473 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" event={"ID":"ec6e14a4-77f8-4698-b2e7-7c6fada680d8","Type":"ContainerStarted","Data":"0c6bc554b479aea824b2d4e6130c830ab152db973500a982dfe4ceff797fdaaa"} Sep 09 21:19:04.776627 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.776532 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" event={"ID":"ec6e14a4-77f8-4698-b2e7-7c6fada680d8","Type":"ContainerStarted","Data":"6bd34e462e7a502a42110bdf3943fe950aaed571ff28f6ec0b07faafb90c019b"} Sep 09 21:19:04.776627 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.776561 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" event={"ID":"ec6e14a4-77f8-4698-b2e7-7c6fada680d8","Type":"ContainerStarted","Data":"ac8769cf3f7ed34854dd43cac4db015c5ceae593159d3ef1476e021169119dd7"} Sep 09 21:19:04.792186 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792156 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792267 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792210 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792312 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792262 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-web-config\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792363 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792309 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20ed06a5-a29d-48ec-b799-f12bcfabcb82-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792363 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792340 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20ed06a5-a29d-48ec-b799-f12bcfabcb82-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792432 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792367 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g9pm\" (UniqueName: \"kubernetes.io/projected/20ed06a5-a29d-48ec-b799-f12bcfabcb82-kube-api-access-9g9pm\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792432 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792390 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792494 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792431 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20ed06a5-a29d-48ec-b799-f12bcfabcb82-tls-assets\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792494 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792458 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792553 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792491 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/20ed06a5-a29d-48ec-b799-f12bcfabcb82-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792553 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792534 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20ed06a5-a29d-48ec-b799-f12bcfabcb82-config-out\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792612 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792591 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.792684 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.792632 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-config-volume\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.893597 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893568 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/20ed06a5-a29d-48ec-b799-f12bcfabcb82-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.893730 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893616 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20ed06a5-a29d-48ec-b799-f12bcfabcb82-config-out\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.893730 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893651 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.893730 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893688 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-config-volume\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.893906 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893757 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.893906 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893791 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.893906 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893821 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-web-config\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.893906 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893865 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20ed06a5-a29d-48ec-b799-f12bcfabcb82-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.893906 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893887 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20ed06a5-a29d-48ec-b799-f12bcfabcb82-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.894140 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893915 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g9pm\" (UniqueName: \"kubernetes.io/projected/20ed06a5-a29d-48ec-b799-f12bcfabcb82-kube-api-access-9g9pm\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.894140 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893943 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.894140 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893971 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20ed06a5-a29d-48ec-b799-f12bcfabcb82-tls-assets\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.894140 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.893994 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.894140 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.894067 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/20ed06a5-a29d-48ec-b799-f12bcfabcb82-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.895403 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.894851 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20ed06a5-a29d-48ec-b799-f12bcfabcb82-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.895403 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.895350 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20ed06a5-a29d-48ec-b799-f12bcfabcb82-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.897973 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.897927 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.898074 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.898018 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.898691 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.898603 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-config-volume\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.898691 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.898670 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.899012 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.898941 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20ed06a5-a29d-48ec-b799-f12bcfabcb82-tls-assets\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.899012 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.898955 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.899012 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.898975 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20ed06a5-a29d-48ec-b799-f12bcfabcb82-config-out\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.899295 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.899272 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.900566 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.900546 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20ed06a5-a29d-48ec-b799-f12bcfabcb82-web-config\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.903103 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.903083 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g9pm\" (UniqueName: \"kubernetes.io/projected/20ed06a5-a29d-48ec-b799-f12bcfabcb82-kube-api-access-9g9pm\") pod \"alertmanager-main-0\" (UID: \"20ed06a5-a29d-48ec-b799-f12bcfabcb82\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:04.998377 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:04.998342 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:19:05.291800 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:05.291732 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 21:19:05.506464 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:19:05.506422 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ed06a5_a29d_48ec_b799_f12bcfabcb82.slice/crio-4561b3b6eb7efd60d79db6c7997c47efd0c83810e0931c10f9515d8aeacdff19 WatchSource:0}: Error finding container 4561b3b6eb7efd60d79db6c7997c47efd0c83810e0931c10f9515d8aeacdff19: Status 404 returned error can't find the container with id 4561b3b6eb7efd60d79db6c7997c47efd0c83810e0931c10f9515d8aeacdff19 Sep 09 21:19:05.781819 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:05.781785 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" event={"ID":"ec6e14a4-77f8-4698-b2e7-7c6fada680d8","Type":"ContainerStarted","Data":"f319445cf7c106c3158ccf910fb6b599ecfc9d24646ff0831605d23089a1c3cc"} Sep 09 21:19:05.783119 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:05.783080 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20ed06a5-a29d-48ec-b799-f12bcfabcb82","Type":"ContainerStarted","Data":"4561b3b6eb7efd60d79db6c7997c47efd0c83810e0931c10f9515d8aeacdff19"} Sep 09 21:19:05.784555 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:05.784532 2550 generic.go:356] "Generic (PLEG): container finished" podID="1a65e9c3-a61b-4ce5-baba-4b101f963a66" containerID="4aa4cd201142158ec13c622558d779b79b8eaf6af3057bb049b5a8c6b5476410" exitCode=0 Sep 09 21:19:05.784651 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:05.784565 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gkwg5" event={"ID":"1a65e9c3-a61b-4ce5-baba-4b101f963a66","Type":"ContainerDied","Data":"4aa4cd201142158ec13c622558d779b79b8eaf6af3057bb049b5a8c6b5476410"} Sep 09 21:19:05.800941 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:05.800886 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-z4chv" podStartSLOduration=1.64004306 podStartE2EDuration="2.800865959s" totalImagesPullingTime="1.160822899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:05.799399471 +0000 UTC m=+548.312136307" watchObservedRunningTime="2026-09-09 21:19:05.800865959 +0000 UTC m=+548.313602779" Sep 09 21:19:06.791227 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:06.791191 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gkwg5" event={"ID":"1a65e9c3-a61b-4ce5-baba-4b101f963a66","Type":"ContainerStarted","Data":"863b98a845a99972320121561fc5f5854af728224e2be120f4540e4a708b262a"} Sep 09 21:19:06.791666 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:06.791246 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gkwg5" event={"ID":"1a65e9c3-a61b-4ce5-baba-4b101f963a66","Type":"ContainerStarted","Data":"9d423f40ab815887ae1364054e360836cdd197fe2497553856966dcb9a9eefe5"} Sep 09 21:19:06.792901 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:06.792877 2550 generic.go:356] "Generic (PLEG): container finished" podID="20ed06a5-a29d-48ec-b799-f12bcfabcb82" containerID="6cabf5601852bca57e296ffb0ff3f0d35657781bf01802a9089943e7c4cf3bf2" exitCode=0 Sep 09 21:19:06.793019 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:06.792967 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20ed06a5-a29d-48ec-b799-f12bcfabcb82","Type":"ContainerDied","Data":"6cabf5601852bca57e296ffb0ff3f0d35657781bf01802a9089943e7c4cf3bf2"} Sep 09 21:19:06.810070 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:06.810018 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gkwg5" podStartSLOduration=2.785762952 podStartE2EDuration="3.810004981s" totalImagesPullingTime="954.978899ms" totalInitContainerRuntime="69.26313ms" isStatefulPod=true podCreationTimestamp="2026-09-09 21:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:06.808972424 +0000 UTC m=+549.321709258" watchObservedRunningTime="2026-09-09 21:19:06.810004981 +0000 UTC m=+549.322741799" Sep 09 21:19:08.054833 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.054797 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-85b67464c9-cmt7t"] Sep 09 21:19:08.059011 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.058987 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.062712 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.062687 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 09 21:19:08.062840 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.062697 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-201m75s53fjln\"" Sep 09 21:19:08.062840 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.062785 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 09 21:19:08.062966 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.062845 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6lqxb\"" Sep 09 21:19:08.063019 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.062964 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 09 21:19:08.063095 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.063081 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 21:19:08.071602 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.071580 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85b67464c9-cmt7t"] Sep 09 21:19:08.119838 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.119810 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d8bf1d0-188f-471c-beeb-4519940b493f-client-ca-bundle\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.119979 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.119847 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8d8bf1d0-188f-471c-beeb-4519940b493f-audit-log\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.119979 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.119874 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsdkf\" (UniqueName: \"kubernetes.io/projected/8d8bf1d0-188f-471c-beeb-4519940b493f-kube-api-access-hsdkf\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.119979 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.119939 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8d8bf1d0-188f-471c-beeb-4519940b493f-secret-metrics-server-tls\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.120117 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.119994 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8d8bf1d0-188f-471c-beeb-4519940b493f-metrics-server-audit-profiles\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.120117 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.120024 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8d8bf1d0-188f-471c-beeb-4519940b493f-secret-metrics-server-client-certs\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.120117 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.120045 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d8bf1d0-188f-471c-beeb-4519940b493f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.220869 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.220831 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d8bf1d0-188f-471c-beeb-4519940b493f-client-ca-bundle\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.220869 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.220874 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8d8bf1d0-188f-471c-beeb-4519940b493f-audit-log\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.221135 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.220900 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsdkf\" (UniqueName: \"kubernetes.io/projected/8d8bf1d0-188f-471c-beeb-4519940b493f-kube-api-access-hsdkf\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.221135 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.220931 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8d8bf1d0-188f-471c-beeb-4519940b493f-secret-metrics-server-tls\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.221135 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.220978 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8d8bf1d0-188f-471c-beeb-4519940b493f-metrics-server-audit-profiles\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.221284 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.221134 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8d8bf1d0-188f-471c-beeb-4519940b493f-secret-metrics-server-client-certs\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.221284 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.221179 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d8bf1d0-188f-471c-beeb-4519940b493f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.221404 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.221306 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8d8bf1d0-188f-471c-beeb-4519940b493f-audit-log\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.222015 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.221988 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d8bf1d0-188f-471c-beeb-4519940b493f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.222283 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.222258 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8d8bf1d0-188f-471c-beeb-4519940b493f-metrics-server-audit-profiles\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.223909 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.223882 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8d8bf1d0-188f-471c-beeb-4519940b493f-secret-metrics-server-client-certs\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.223909 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.223901 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d8bf1d0-188f-471c-beeb-4519940b493f-client-ca-bundle\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.224464 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.224441 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8d8bf1d0-188f-471c-beeb-4519940b493f-secret-metrics-server-tls\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.229880 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.229860 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsdkf\" (UniqueName: \"kubernetes.io/projected/8d8bf1d0-188f-471c-beeb-4519940b493f-kube-api-access-hsdkf\") pod \"metrics-server-85b67464c9-cmt7t\" (UID: \"8d8bf1d0-188f-471c-beeb-4519940b493f\") " pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.370678 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.370650 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:08.787728 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.787699 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85b67464c9-cmt7t"] Sep 09 21:19:08.792064 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:19:08.792002 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d8bf1d0_188f_471c_beeb_4519940b493f.slice/crio-e061c4aeeea5bba7270bd4987b7e4b83d44d331a04bfdfad710baefe1ea1a938 WatchSource:0}: Error finding container e061c4aeeea5bba7270bd4987b7e4b83d44d331a04bfdfad710baefe1ea1a938: Status 404 returned error can't find the container with id e061c4aeeea5bba7270bd4987b7e4b83d44d331a04bfdfad710baefe1ea1a938 Sep 09 21:19:08.802371 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.802339 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" event={"ID":"8d8bf1d0-188f-471c-beeb-4519940b493f","Type":"ContainerStarted","Data":"e061c4aeeea5bba7270bd4987b7e4b83d44d331a04bfdfad710baefe1ea1a938"} Sep 09 21:19:08.805055 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.805033 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20ed06a5-a29d-48ec-b799-f12bcfabcb82","Type":"ContainerStarted","Data":"a86f0ae7511e960d4f3395c0e47e5026ffa7271d1b990ca6f41153984d65b557"} Sep 09 21:19:08.805135 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.805086 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20ed06a5-a29d-48ec-b799-f12bcfabcb82","Type":"ContainerStarted","Data":"8ad7faa9ef284445dd9bbb0271943b5e2c43859bbcefc091b518be8ab4b695c8"} Sep 09 21:19:08.805135 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:08.805112 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20ed06a5-a29d-48ec-b799-f12bcfabcb82","Type":"ContainerStarted","Data":"a5fccdf1123b303044345e7ae9be94da5220b3fe883126d6c61553f248a9fd5d"} Sep 09 21:19:09.814088 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:09.814046 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20ed06a5-a29d-48ec-b799-f12bcfabcb82","Type":"ContainerStarted","Data":"e4f0dfd25b858a3aa4cab7cc9dfbf7c1e0b5726b22e60b54b399a691f755c3c0"} Sep 09 21:19:09.814530 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:09.814131 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20ed06a5-a29d-48ec-b799-f12bcfabcb82","Type":"ContainerStarted","Data":"88896b66cefd3a4ca7f4f626afcb1ec4654e8d3e3352b06e1d3379bfbe15bb0f"} Sep 09 21:19:10.819499 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:10.819459 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" event={"ID":"8d8bf1d0-188f-471c-beeb-4519940b493f","Type":"ContainerStarted","Data":"45647c613df584e3b59123f0105ccd51f9cd8c14bfc571d4491d4e91978ef193"} Sep 09 21:19:10.822640 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:10.822611 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20ed06a5-a29d-48ec-b799-f12bcfabcb82","Type":"ContainerStarted","Data":"96ba99b23d7996ecd8531db22ddf58d8e172340fcd30c2761aab9c591d34c750"} Sep 09 21:19:10.841901 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:10.841859 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" podStartSLOduration=1.461776465 podStartE2EDuration="2.841844717s" totalImagesPullingTime="1.380068252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:19:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:10.840529705 +0000 UTC m=+553.353266573" watchObservedRunningTime="2026-09-09 21:19:10.841844717 +0000 UTC m=+553.354581533" Sep 09 21:19:10.869134 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:10.869089 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.933845225 podStartE2EDuration="6.869077452s" totalImagesPullingTime="3.919603073s" totalInitContainerRuntime="15.629154ms" isStatefulPod=false podCreationTimestamp="2026-09-09 21:19:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:10.868188762 +0000 UTC m=+553.380925628" watchObservedRunningTime="2026-09-09 21:19:10.869077452 +0000 UTC m=+553.381814271" Sep 09 21:19:16.744820 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:16.744787 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d965ffd6c-xsjkg" Sep 09 21:19:18.627590 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:18.627540 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-864846d867-ptrds" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" containerID="cri-o://b9c1089b7819058b31e728a2514649abe434b43f84b24f7357a9719174be9751" gracePeriod=30 Sep 09 21:19:18.856143 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:18.856113 2550 generic.go:356] "Generic (PLEG): container finished" podID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerID="b9c1089b7819058b31e728a2514649abe434b43f84b24f7357a9719174be9751" exitCode=0 Sep 09 21:19:18.856272 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:18.856169 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerDied","Data":"b9c1089b7819058b31e728a2514649abe434b43f84b24f7357a9719174be9751"} Sep 09 21:19:18.856272 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:18.856211 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864846d867-ptrds" event={"ID":"eca72d7b-7575-4cec-881d-d1d9ae20f872","Type":"ContainerDied","Data":"19ffa46396440f63de69427e73e34af5f08da5b86f1153f42743459128004c01"} Sep 09 21:19:18.856272 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:18.856229 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19ffa46396440f63de69427e73e34af5f08da5b86f1153f42743459128004c01" Sep 09 21:19:18.856272 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:18.856245 2550 scope.go:117] "RemoveContainer" containerID="c4ec1b6a4668a1dadc9464742440799548f91b130bacbfacf5a7081030985066" Sep 09 21:19:18.866334 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:18.866302 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:19:19.008269 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.008167 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-image-registry-private-configuration\") pod \"eca72d7b-7575-4cec-881d-d1d9ae20f872\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " Sep 09 21:19:19.008269 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.008223 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-trusted-ca\") pod \"eca72d7b-7575-4cec-881d-d1d9ae20f872\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " Sep 09 21:19:19.008517 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.008306 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-bound-sa-token\") pod \"eca72d7b-7575-4cec-881d-d1d9ae20f872\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " Sep 09 21:19:19.008517 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.008354 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls\") pod \"eca72d7b-7575-4cec-881d-d1d9ae20f872\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " Sep 09 21:19:19.008517 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.008391 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eca72d7b-7575-4cec-881d-d1d9ae20f872-ca-trust-extracted\") pod \"eca72d7b-7575-4cec-881d-d1d9ae20f872\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " Sep 09 21:19:19.008517 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.008428 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-certificates\") pod \"eca72d7b-7575-4cec-881d-d1d9ae20f872\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " Sep 09 21:19:19.008517 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.008462 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gshcw\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-kube-api-access-gshcw\") pod \"eca72d7b-7575-4cec-881d-d1d9ae20f872\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " Sep 09 21:19:19.008517 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.008505 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-installation-pull-secrets\") pod \"eca72d7b-7575-4cec-881d-d1d9ae20f872\" (UID: \"eca72d7b-7575-4cec-881d-d1d9ae20f872\") " Sep 09 21:19:19.008819 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.008791 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "eca72d7b-7575-4cec-881d-d1d9ae20f872" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:19.009415 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.009375 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "eca72d7b-7575-4cec-881d-d1d9ae20f872" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:19.011164 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.011109 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "eca72d7b-7575-4cec-881d-d1d9ae20f872" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:19:19.011164 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.011134 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "eca72d7b-7575-4cec-881d-d1d9ae20f872" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:19:19.011359 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.011296 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-kube-api-access-gshcw" (OuterVolumeSpecName: "kube-api-access-gshcw") pod "eca72d7b-7575-4cec-881d-d1d9ae20f872" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872"). InnerVolumeSpecName "kube-api-access-gshcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:19:19.011444 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.011377 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "eca72d7b-7575-4cec-881d-d1d9ae20f872" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:19:19.011480 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.011456 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "eca72d7b-7575-4cec-881d-d1d9ae20f872" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:19:19.017640 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.017617 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eca72d7b-7575-4cec-881d-d1d9ae20f872-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "eca72d7b-7575-4cec-881d-d1d9ae20f872" (UID: "eca72d7b-7575-4cec-881d-d1d9ae20f872"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:19:19.039216 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039195 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57dd9ccdd9-v4hqp"] Sep 09 21:19:19.039501 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039488 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039549 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039505 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039549 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039512 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039549 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039517 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039549 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039523 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039549 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039528 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039549 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039537 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039549 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039542 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039549 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039547 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039549 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039552 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039802 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039607 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039802 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039613 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039802 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039620 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039802 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039627 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.039802 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.039633 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:19:19.044200 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.044181 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.047463 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.047448 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 09 21:19:19.047810 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.047796 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 09 21:19:19.047922 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.047904 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 09 21:19:19.047982 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.047931 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 09 21:19:19.048044 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.048033 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 09 21:19:19.048237 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.048222 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9pr5v\"" Sep 09 21:19:19.048309 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.048224 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 09 21:19:19.048482 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.048468 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 09 21:19:19.053696 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.053679 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 09 21:19:19.056312 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.056292 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57dd9ccdd9-v4hqp"] Sep 09 21:19:19.110138 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110102 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-console-config\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.110351 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110158 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlhxv\" (UniqueName: \"kubernetes.io/projected/ffd276a1-6cba-4890-99c4-aeba45b14242-kube-api-access-mlhxv\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.110351 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110223 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-serving-cert\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.110351 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110271 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-oauth-serving-cert\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.110351 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110311 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-oauth-config\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.110497 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110373 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-trusted-ca-bundle\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.110497 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110405 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-service-ca\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.110497 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110463 2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-installation-pull-secrets\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:19:19.110497 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110482 2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eca72d7b-7575-4cec-881d-d1d9ae20f872-image-registry-private-configuration\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:19:19.110497 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110493 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-trusted-ca\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:19:19.110640 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110501 2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-bound-sa-token\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:19:19.110640 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110509 2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-tls\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:19:19.110640 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110516 2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eca72d7b-7575-4cec-881d-d1d9ae20f872-ca-trust-extracted\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:19:19.110640 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110524 2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eca72d7b-7575-4cec-881d-d1d9ae20f872-registry-certificates\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:19:19.110640 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.110532 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gshcw\" (UniqueName: \"kubernetes.io/projected/eca72d7b-7575-4cec-881d-d1d9ae20f872-kube-api-access-gshcw\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:19:19.211555 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.211527 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-oauth-config\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.211672 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.211562 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-trusted-ca-bundle\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.211672 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.211587 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-service-ca\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.211672 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.211614 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-console-config\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.211672 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.211650 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlhxv\" (UniqueName: \"kubernetes.io/projected/ffd276a1-6cba-4890-99c4-aeba45b14242-kube-api-access-mlhxv\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.211844 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.211676 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-serving-cert\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.211844 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.211708 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-oauth-serving-cert\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.212466 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.212442 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-service-ca\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.212562 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.212540 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-oauth-serving-cert\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.212629 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.212614 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-trusted-ca-bundle\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.212821 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.212801 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-console-config\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.214135 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.214115 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-oauth-config\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.214261 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.214242 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-serving-cert\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.220289 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.220272 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlhxv\" (UniqueName: \"kubernetes.io/projected/ffd276a1-6cba-4890-99c4-aeba45b14242-kube-api-access-mlhxv\") pod \"console-57dd9ccdd9-v4hqp\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.353488 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.353414 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:19.626374 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.626171 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57dd9ccdd9-v4hqp"] Sep 09 21:19:19.628800 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:19:19.628775 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd276a1_6cba_4890_99c4_aeba45b14242.slice/crio-db73b56dbcf7afe35cef7f34b160e070424b5eea399c8b1458ed5c3f734025fe WatchSource:0}: Error finding container db73b56dbcf7afe35cef7f34b160e070424b5eea399c8b1458ed5c3f734025fe: Status 404 returned error can't find the container with id db73b56dbcf7afe35cef7f34b160e070424b5eea399c8b1458ed5c3f734025fe Sep 09 21:19:19.860899 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.860875 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864846d867-ptrds" Sep 09 21:19:19.862206 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.862176 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57dd9ccdd9-v4hqp" event={"ID":"ffd276a1-6cba-4890-99c4-aeba45b14242","Type":"ContainerStarted","Data":"db73b56dbcf7afe35cef7f34b160e070424b5eea399c8b1458ed5c3f734025fe"} Sep 09 21:19:19.895900 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.895829 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-864846d867-ptrds"] Sep 09 21:19:19.902176 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.902156 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-864846d867-ptrds"] Sep 09 21:19:19.994277 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:19.994244 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" path="/var/lib/kubelet/pods/eca72d7b-7575-4cec-881d-d1d9ae20f872/volumes" Sep 09 21:19:22.876355 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:22.876305 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57dd9ccdd9-v4hqp" event={"ID":"ffd276a1-6cba-4890-99c4-aeba45b14242","Type":"ContainerStarted","Data":"a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215"} Sep 09 21:19:22.897767 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:22.897700 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57dd9ccdd9-v4hqp" podStartSLOduration=1.371680889 podStartE2EDuration="3.897683579s" totalImagesPullingTime="2.52600269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:22.89625237 +0000 UTC m=+565.408989203" watchObservedRunningTime="2026-09-09 21:19:22.897683579 +0000 UTC m=+565.410420398" Sep 09 21:19:28.371109 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:28.371079 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:28.371567 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:28.371132 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:29.354361 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:29.354331 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:29.354550 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:29.354387 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:29.359143 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:29.359122 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:29.906204 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:29.906174 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:19:48.376608 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:48.376578 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:19:48.380281 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:19:48.380264 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-85b67464c9-cmt7t" Sep 09 21:20:29.224795 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.224762 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-777cbdc68d-fltgx"] Sep 09 21:20:29.226719 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.225068 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:20:29.226719 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.225080 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:20:29.226719 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.225090 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:20:29.226719 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.225095 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:20:29.226719 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.225149 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:20:29.226719 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.225156 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca72d7b-7575-4cec-881d-d1d9ae20f872" containerName="registry" Sep 09 21:20:29.227520 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.227504 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.238270 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.238245 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-777cbdc68d-fltgx"] Sep 09 21:20:29.360868 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.360827 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-oauth-serving-cert\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.360868 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.360874 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbrnx\" (UniqueName: \"kubernetes.io/projected/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-kube-api-access-mbrnx\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.361114 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.360955 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-service-ca\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.361114 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.361008 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-oauth-config\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.361114 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.361030 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-trusted-ca-bundle\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.361114 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.361067 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-serving-cert\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.361114 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.361094 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-config\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.461925 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.461883 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-serving-cert\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.462101 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.461934 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-config\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.462101 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.461957 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-oauth-serving-cert\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.462101 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.461982 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbrnx\" (UniqueName: \"kubernetes.io/projected/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-kube-api-access-mbrnx\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.462255 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.462125 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-service-ca\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.462255 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.462192 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-oauth-config\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.462255 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.462224 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-trusted-ca-bundle\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.462815 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.462729 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-config\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.462815 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.462842 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-oauth-serving-cert\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.463064 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.462844 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-service-ca\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.463123 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.463094 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-trusted-ca-bundle\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.464790 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.464764 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-serving-cert\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.464874 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.464814 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-oauth-config\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.470998 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.470976 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbrnx\" (UniqueName: \"kubernetes.io/projected/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-kube-api-access-mbrnx\") pod \"console-777cbdc68d-fltgx\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.537944 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.537837 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:29.814181 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:29.814157 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-777cbdc68d-fltgx"] Sep 09 21:20:29.816877 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:20:29.816849 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod677cf65a_ce58_40b7_9ecb_3840a75e1d9f.slice/crio-13435a1129da4c50d56422b158af380ec1bc0dd2ea6823dc07ff0fd935f7c948 WatchSource:0}: Error finding container 13435a1129da4c50d56422b158af380ec1bc0dd2ea6823dc07ff0fd935f7c948: Status 404 returned error can't find the container with id 13435a1129da4c50d56422b158af380ec1bc0dd2ea6823dc07ff0fd935f7c948 Sep 09 21:20:30.130682 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:30.130648 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777cbdc68d-fltgx" event={"ID":"677cf65a-ce58-40b7-9ecb-3840a75e1d9f","Type":"ContainerStarted","Data":"b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552"} Sep 09 21:20:30.130832 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:30.130698 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777cbdc68d-fltgx" event={"ID":"677cf65a-ce58-40b7-9ecb-3840a75e1d9f","Type":"ContainerStarted","Data":"13435a1129da4c50d56422b158af380ec1bc0dd2ea6823dc07ff0fd935f7c948"} Sep 09 21:20:30.149014 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:30.148954 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-777cbdc68d-fltgx" podStartSLOduration=1.14893805 podStartE2EDuration="1.14893805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:20:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:20:30.147911176 +0000 UTC m=+632.660647995" watchObservedRunningTime="2026-09-09 21:20:30.14893805 +0000 UTC m=+632.661674869" Sep 09 21:20:39.538144 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:39.538106 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:39.538757 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:39.538251 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:39.543135 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:39.543112 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:40.172412 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:40.172387 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:20:40.221480 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:20:40.221444 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57dd9ccdd9-v4hqp"] Sep 09 21:21:05.241793 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.241732 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57dd9ccdd9-v4hqp" podUID="ffd276a1-6cba-4890-99c4-aeba45b14242" containerName="console" containerID="cri-o://a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215" gracePeriod=15 Sep 09 21:21:05.494247 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.494170 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:21:05.562212 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.562170 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-trusted-ca-bundle\") pod \"ffd276a1-6cba-4890-99c4-aeba45b14242\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " Sep 09 21:21:05.562212 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.562222 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-oauth-serving-cert\") pod \"ffd276a1-6cba-4890-99c4-aeba45b14242\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " Sep 09 21:21:05.562514 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.562254 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-console-config\") pod \"ffd276a1-6cba-4890-99c4-aeba45b14242\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " Sep 09 21:21:05.562514 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.562290 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-oauth-config\") pod \"ffd276a1-6cba-4890-99c4-aeba45b14242\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " Sep 09 21:21:05.562514 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.562314 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlhxv\" (UniqueName: \"kubernetes.io/projected/ffd276a1-6cba-4890-99c4-aeba45b14242-kube-api-access-mlhxv\") pod \"ffd276a1-6cba-4890-99c4-aeba45b14242\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " Sep 09 21:21:05.562514 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.562390 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-service-ca\") pod \"ffd276a1-6cba-4890-99c4-aeba45b14242\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " Sep 09 21:21:05.562514 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.562422 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-serving-cert\") pod \"ffd276a1-6cba-4890-99c4-aeba45b14242\" (UID: \"ffd276a1-6cba-4890-99c4-aeba45b14242\") " Sep 09 21:21:05.562839 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.562788 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ffd276a1-6cba-4890-99c4-aeba45b14242" (UID: "ffd276a1-6cba-4890-99c4-aeba45b14242"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:21:05.562892 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.562850 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ffd276a1-6cba-4890-99c4-aeba45b14242" (UID: "ffd276a1-6cba-4890-99c4-aeba45b14242"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:21:05.562892 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.562871 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-console-config" (OuterVolumeSpecName: "console-config") pod "ffd276a1-6cba-4890-99c4-aeba45b14242" (UID: "ffd276a1-6cba-4890-99c4-aeba45b14242"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:21:05.562892 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.562864 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-service-ca" (OuterVolumeSpecName: "service-ca") pod "ffd276a1-6cba-4890-99c4-aeba45b14242" (UID: "ffd276a1-6cba-4890-99c4-aeba45b14242"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:21:05.564914 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.564880 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ffd276a1-6cba-4890-99c4-aeba45b14242" (UID: "ffd276a1-6cba-4890-99c4-aeba45b14242"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:21:05.564914 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.564896 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ffd276a1-6cba-4890-99c4-aeba45b14242" (UID: "ffd276a1-6cba-4890-99c4-aeba45b14242"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:21:05.564914 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.564891 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffd276a1-6cba-4890-99c4-aeba45b14242-kube-api-access-mlhxv" (OuterVolumeSpecName: "kube-api-access-mlhxv") pod "ffd276a1-6cba-4890-99c4-aeba45b14242" (UID: "ffd276a1-6cba-4890-99c4-aeba45b14242"). InnerVolumeSpecName "kube-api-access-mlhxv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:21:05.663648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.663603 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-serving-cert\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:05.663648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.663641 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-trusted-ca-bundle\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:05.663648 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.663657 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-oauth-serving-cert\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:05.663904 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.663669 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-console-config\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:05.663904 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.663680 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffd276a1-6cba-4890-99c4-aeba45b14242-console-oauth-config\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:05.663904 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.663692 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlhxv\" (UniqueName: \"kubernetes.io/projected/ffd276a1-6cba-4890-99c4-aeba45b14242-kube-api-access-mlhxv\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:05.663904 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:05.663702 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd276a1-6cba-4890-99c4-aeba45b14242-service-ca\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:06.267501 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:06.267407 2550 generic.go:356] "Generic (PLEG): container finished" podID="ffd276a1-6cba-4890-99c4-aeba45b14242" containerID="a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215" exitCode=0 Sep 09 21:21:06.267501 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:06.267455 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57dd9ccdd9-v4hqp" Sep 09 21:21:06.267929 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:06.267495 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57dd9ccdd9-v4hqp" event={"ID":"ffd276a1-6cba-4890-99c4-aeba45b14242","Type":"ContainerDied","Data":"a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215"} Sep 09 21:21:06.267929 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:06.267544 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57dd9ccdd9-v4hqp" event={"ID":"ffd276a1-6cba-4890-99c4-aeba45b14242","Type":"ContainerDied","Data":"db73b56dbcf7afe35cef7f34b160e070424b5eea399c8b1458ed5c3f734025fe"} Sep 09 21:21:06.267929 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:06.267564 2550 scope.go:117] "RemoveContainer" containerID="a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215" Sep 09 21:21:06.277675 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:06.277649 2550 scope.go:117] "RemoveContainer" containerID="a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215" Sep 09 21:21:06.278027 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:21:06.278002 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215\": container with ID starting with a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215 not found: ID does not exist" containerID="a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215" Sep 09 21:21:06.278088 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:06.278033 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215"} err="failed to get container status \"a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215\": rpc error: code = NotFound desc = could not find container \"a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215\": container with ID starting with a550030188853a3dbd5584aa562a1c54d67c79ba73080923346516389dd35215 not found: ID does not exist" Sep 09 21:21:06.285937 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:06.285910 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57dd9ccdd9-v4hqp"] Sep 09 21:21:06.289485 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:06.289461 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57dd9ccdd9-v4hqp"] Sep 09 21:21:07.990484 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:07.990451 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffd276a1-6cba-4890-99c4-aeba45b14242" path="/var/lib/kubelet/pods/ffd276a1-6cba-4890-99c4-aeba45b14242/volumes" Sep 09 21:21:16.001075 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.001048 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-865cfd664b-8pjwv"] Sep 09 21:21:16.001544 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.001379 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffd276a1-6cba-4890-99c4-aeba45b14242" containerName="console" Sep 09 21:21:16.001544 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.001391 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffd276a1-6cba-4890-99c4-aeba45b14242" containerName="console" Sep 09 21:21:16.001544 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.001439 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffd276a1-6cba-4890-99c4-aeba45b14242" containerName="console" Sep 09 21:21:16.005742 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.005723 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.018499 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.018474 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-865cfd664b-8pjwv"] Sep 09 21:21:16.046865 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.046824 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc8fx\" (UniqueName: \"kubernetes.io/projected/1ce39040-5a9b-47a5-a433-2a33ad972b99-kube-api-access-kc8fx\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.047054 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.046875 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-service-ca\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.047054 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.046895 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-oauth-serving-cert\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.047054 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.046942 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-console-config\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.047054 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.046987 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce39040-5a9b-47a5-a433-2a33ad972b99-console-serving-cert\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.047054 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.047047 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ce39040-5a9b-47a5-a433-2a33ad972b99-console-oauth-config\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.047234 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.047070 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-trusted-ca-bundle\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.147619 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.147579 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ce39040-5a9b-47a5-a433-2a33ad972b99-console-oauth-config\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.147619 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.147619 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-trusted-ca-bundle\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.147869 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.147649 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc8fx\" (UniqueName: \"kubernetes.io/projected/1ce39040-5a9b-47a5-a433-2a33ad972b99-kube-api-access-kc8fx\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.147869 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.147776 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-service-ca\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.147869 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.147825 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-oauth-serving-cert\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.147869 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.147855 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-console-config\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.148068 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.147888 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce39040-5a9b-47a5-a433-2a33ad972b99-console-serving-cert\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.148605 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.148583 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-trusted-ca-bundle\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.148704 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.148591 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-service-ca\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.148704 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.148617 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-oauth-serving-cert\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.148774 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.148748 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ce39040-5a9b-47a5-a433-2a33ad972b99-console-config\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.150382 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.150361 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ce39040-5a9b-47a5-a433-2a33ad972b99-console-oauth-config\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.150507 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.150491 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce39040-5a9b-47a5-a433-2a33ad972b99-console-serving-cert\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.156397 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.156375 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc8fx\" (UniqueName: \"kubernetes.io/projected/1ce39040-5a9b-47a5-a433-2a33ad972b99-kube-api-access-kc8fx\") pod \"console-865cfd664b-8pjwv\" (UID: \"1ce39040-5a9b-47a5-a433-2a33ad972b99\") " pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.314474 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.314393 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:16.588090 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:16.588011 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-865cfd664b-8pjwv"] Sep 09 21:21:16.591218 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:21:16.591192 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ce39040_5a9b_47a5_a433_2a33ad972b99.slice/crio-1b7af556faf0ec57a125414db6ace6ff03e1477a51ceac973d7fa0a624e9aa3f WatchSource:0}: Error finding container 1b7af556faf0ec57a125414db6ace6ff03e1477a51ceac973d7fa0a624e9aa3f: Status 404 returned error can't find the container with id 1b7af556faf0ec57a125414db6ace6ff03e1477a51ceac973d7fa0a624e9aa3f Sep 09 21:21:17.311559 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:17.311521 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-865cfd664b-8pjwv" event={"ID":"1ce39040-5a9b-47a5-a433-2a33ad972b99","Type":"ContainerStarted","Data":"4b3b375e7bf7a2b5e42f3a6cb85e698405fd96937f9e1cff1ea2c6160ea1bf16"} Sep 09 21:21:17.311948 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:17.311601 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-865cfd664b-8pjwv" event={"ID":"1ce39040-5a9b-47a5-a433-2a33ad972b99","Type":"ContainerStarted","Data":"1b7af556faf0ec57a125414db6ace6ff03e1477a51ceac973d7fa0a624e9aa3f"} Sep 09 21:21:17.328480 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:17.328434 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-865cfd664b-8pjwv" podStartSLOduration=2.328418703 podStartE2EDuration="2.328418703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:21:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:21:17.327473417 +0000 UTC m=+679.840210248" watchObservedRunningTime="2026-09-09 21:21:17.328418703 +0000 UTC m=+679.841155521" Sep 09 21:21:26.315137 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:26.315086 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:26.315569 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:26.315190 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:26.320112 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:26.320088 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:26.352143 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:26.352118 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-865cfd664b-8pjwv" Sep 09 21:21:26.402147 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:26.402117 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-777cbdc68d-fltgx"] Sep 09 21:21:51.422421 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.422307 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-777cbdc68d-fltgx" podUID="677cf65a-ce58-40b7-9ecb-3840a75e1d9f" containerName="console" containerID="cri-o://b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552" gracePeriod=15 Sep 09 21:21:51.656927 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.656903 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:21:51.824065 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.824000 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-oauth-config\") pod \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " Sep 09 21:21:51.824065 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.824042 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-serving-cert\") pod \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " Sep 09 21:21:51.824276 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.824068 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-service-ca\") pod \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " Sep 09 21:21:51.824276 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.824094 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-config\") pod \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " Sep 09 21:21:51.824276 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.824206 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-oauth-serving-cert\") pod \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " Sep 09 21:21:51.824276 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.824233 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-trusted-ca-bundle\") pod \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " Sep 09 21:21:51.824276 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.824258 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbrnx\" (UniqueName: \"kubernetes.io/projected/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-kube-api-access-mbrnx\") pod \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\" (UID: \"677cf65a-ce58-40b7-9ecb-3840a75e1d9f\") " Sep 09 21:21:51.824568 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.824503 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-service-ca" (OuterVolumeSpecName: "service-ca") pod "677cf65a-ce58-40b7-9ecb-3840a75e1d9f" (UID: "677cf65a-ce58-40b7-9ecb-3840a75e1d9f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:21:51.824568 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.824511 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-config" (OuterVolumeSpecName: "console-config") pod "677cf65a-ce58-40b7-9ecb-3840a75e1d9f" (UID: "677cf65a-ce58-40b7-9ecb-3840a75e1d9f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:21:51.824664 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.824625 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "677cf65a-ce58-40b7-9ecb-3840a75e1d9f" (UID: "677cf65a-ce58-40b7-9ecb-3840a75e1d9f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:21:51.824664 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.824646 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "677cf65a-ce58-40b7-9ecb-3840a75e1d9f" (UID: "677cf65a-ce58-40b7-9ecb-3840a75e1d9f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:21:51.826505 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.826474 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "677cf65a-ce58-40b7-9ecb-3840a75e1d9f" (UID: "677cf65a-ce58-40b7-9ecb-3840a75e1d9f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:21:51.826505 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.826490 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "677cf65a-ce58-40b7-9ecb-3840a75e1d9f" (UID: "677cf65a-ce58-40b7-9ecb-3840a75e1d9f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:21:51.826635 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.826545 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-kube-api-access-mbrnx" (OuterVolumeSpecName: "kube-api-access-mbrnx") pod "677cf65a-ce58-40b7-9ecb-3840a75e1d9f" (UID: "677cf65a-ce58-40b7-9ecb-3840a75e1d9f"). InnerVolumeSpecName "kube-api-access-mbrnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:21:51.925436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.925405 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-oauth-config\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:51.925436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.925436 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-serving-cert\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:51.925436 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.925446 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-service-ca\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:51.925597 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.925454 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-console-config\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:51.925597 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.925462 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-oauth-serving-cert\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:51.925597 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.925469 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-trusted-ca-bundle\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:51.925597 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:51.925477 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbrnx\" (UniqueName: \"kubernetes.io/projected/677cf65a-ce58-40b7-9ecb-3840a75e1d9f-kube-api-access-mbrnx\") on node \"ip-10-0-128-60.ec2.internal\" DevicePath \"\"" Sep 09 21:21:52.448099 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:52.448068 2550 generic.go:356] "Generic (PLEG): container finished" podID="677cf65a-ce58-40b7-9ecb-3840a75e1d9f" containerID="b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552" exitCode=0 Sep 09 21:21:52.448513 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:52.448117 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777cbdc68d-fltgx" Sep 09 21:21:52.448513 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:52.448162 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777cbdc68d-fltgx" event={"ID":"677cf65a-ce58-40b7-9ecb-3840a75e1d9f","Type":"ContainerDied","Data":"b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552"} Sep 09 21:21:52.448513 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:52.448216 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777cbdc68d-fltgx" event={"ID":"677cf65a-ce58-40b7-9ecb-3840a75e1d9f","Type":"ContainerDied","Data":"13435a1129da4c50d56422b158af380ec1bc0dd2ea6823dc07ff0fd935f7c948"} Sep 09 21:21:52.448513 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:52.448241 2550 scope.go:117] "RemoveContainer" containerID="b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552" Sep 09 21:21:52.456286 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:52.456269 2550 scope.go:117] "RemoveContainer" containerID="b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552" Sep 09 21:21:52.456597 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:21:52.456579 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552\": container with ID starting with b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552 not found: ID does not exist" containerID="b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552" Sep 09 21:21:52.456657 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:52.456605 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552"} err="failed to get container status \"b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552\": rpc error: code = NotFound desc = could not find container \"b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552\": container with ID starting with b1d663f050178566011819f1810545a29e6e79900384668cd8961f92ddaaf552 not found: ID does not exist" Sep 09 21:21:52.474207 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:52.474186 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-777cbdc68d-fltgx"] Sep 09 21:21:52.480257 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:52.480238 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-777cbdc68d-fltgx"] Sep 09 21:21:53.989566 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:21:53.989538 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="677cf65a-ce58-40b7-9ecb-3840a75e1d9f" path="/var/lib/kubelet/pods/677cf65a-ce58-40b7-9ecb-3840a75e1d9f/volumes" Sep 09 21:24:57.917896 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:24:57.917828 2550 scope.go:117] "RemoveContainer" containerID="b9c1089b7819058b31e728a2514649abe434b43f84b24f7357a9719174be9751" Sep 09 21:25:32.402564 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.402528 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v"] Sep 09 21:25:32.404941 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.402805 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="677cf65a-ce58-40b7-9ecb-3840a75e1d9f" containerName="console" Sep 09 21:25:32.404941 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.402815 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="677cf65a-ce58-40b7-9ecb-3840a75e1d9f" containerName="console" Sep 09 21:25:32.404941 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.402870 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="677cf65a-ce58-40b7-9ecb-3840a75e1d9f" containerName="console" Sep 09 21:25:32.405839 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.405824 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.409204 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.409183 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7vrpf\"" Sep 09 21:25:32.409204 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.409193 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 09 21:25:32.409465 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.409236 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 09 21:25:32.409465 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.409296 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 09 21:25:32.409465 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.409331 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 09 21:25:32.423032 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.423012 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v"] Sep 09 21:25:32.514400 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.514373 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.514528 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.514406 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-manifests\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.514528 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.514434 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-tmp\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.514528 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.514514 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-webhook-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.514685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.514604 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vjhz\" (UniqueName: \"kubernetes.io/projected/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-kube-api-access-2vjhz\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.615116 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.615092 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-webhook-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.615214 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.615130 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vjhz\" (UniqueName: \"kubernetes.io/projected/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-kube-api-access-2vjhz\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.615214 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.615183 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.615295 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.615215 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-manifests\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.615295 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.615247 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-tmp\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.615682 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.615659 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-manifests\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.617741 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.617715 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-tmp\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.617831 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.617784 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.617999 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.617978 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-webhook-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.627946 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.627925 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vjhz\" (UniqueName: \"kubernetes.io/projected/44e2c50e-81fd-4d6d-95a1-b441ade7cdd9-kube-api-access-2vjhz\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nml7v\" (UID: \"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:32.716005 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:32.715949 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:33.003437 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:33.003400 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v"] Sep 09 21:25:33.006897 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:25:33.006866 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e2c50e_81fd_4d6d_95a1_b441ade7cdd9.slice/crio-471f3d43532ef1f414e62996d4acdd3040b19dde186ac5eba86aadf38af97d30 WatchSource:0}: Error finding container 471f3d43532ef1f414e62996d4acdd3040b19dde186ac5eba86aadf38af97d30: Status 404 returned error can't find the container with id 471f3d43532ef1f414e62996d4acdd3040b19dde186ac5eba86aadf38af97d30 Sep 09 21:25:33.008013 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:33.007996 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:25:33.266703 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:33.266613 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" event={"ID":"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9","Type":"ContainerStarted","Data":"471f3d43532ef1f414e62996d4acdd3040b19dde186ac5eba86aadf38af97d30"} Sep 09 21:25:36.283430 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:36.283389 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" event={"ID":"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9","Type":"ContainerStarted","Data":"df273660766454628d4f0463a2c37bb9b2d67dbbea4faa1c45b66f1b709e5536"} Sep 09 21:25:37.289411 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:37.289371 2550 generic.go:356] "Generic (PLEG): container finished" podID="44e2c50e-81fd-4d6d-95a1-b441ade7cdd9" containerID="df273660766454628d4f0463a2c37bb9b2d67dbbea4faa1c45b66f1b709e5536" exitCode=0 Sep 09 21:25:37.289824 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:37.289444 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" event={"ID":"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9","Type":"ContainerDied","Data":"df273660766454628d4f0463a2c37bb9b2d67dbbea4faa1c45b66f1b709e5536"} Sep 09 21:25:38.295814 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:38.295776 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" event={"ID":"44e2c50e-81fd-4d6d-95a1-b441ade7cdd9","Type":"ContainerStarted","Data":"080fbbdcef846e4dfa8a27bed01866dbe46543546670b4c449627283626aec25"} Sep 09 21:25:38.296221 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:38.295941 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:38.317828 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:38.317778 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" podStartSLOduration=1.824846291 podStartE2EDuration="6.317757584s" totalImagesPullingTime="3.419424992s" totalInitContainerRuntime="1.073486301s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:25:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:25:38.315598521 +0000 UTC m=+940.828335352" watchObservedRunningTime="2026-09-09 21:25:38.317757584 +0000 UTC m=+940.830494417" Sep 09 21:25:49.301608 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.301578 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nml7v" Sep 09 21:25:49.660278 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.660240 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj"] Sep 09 21:25:49.679472 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.679444 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj"] Sep 09 21:25:49.679611 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.679545 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.682355 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.682313 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 09 21:25:49.682488 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.682369 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bpd92\"" Sep 09 21:25:49.682488 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.682452 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 09 21:25:49.682593 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.682484 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 09 21:25:49.682634 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.682589 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:25:49.682953 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.682935 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 09 21:25:49.749082 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.749055 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9972ef46-4061-4afd-87ed-255e2f89cb1c-metrics-cert\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.749192 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.749095 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9972ef46-4061-4afd-87ed-255e2f89cb1c-manager-config\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.749192 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.749157 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bz2p\" (UniqueName: \"kubernetes.io/projected/9972ef46-4061-4afd-87ed-255e2f89cb1c-kube-api-access-2bz2p\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.749192 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.749186 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9972ef46-4061-4afd-87ed-255e2f89cb1c-cert\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.850446 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.850423 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bz2p\" (UniqueName: \"kubernetes.io/projected/9972ef46-4061-4afd-87ed-255e2f89cb1c-kube-api-access-2bz2p\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.850532 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.850454 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9972ef46-4061-4afd-87ed-255e2f89cb1c-cert\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.850577 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.850517 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9972ef46-4061-4afd-87ed-255e2f89cb1c-metrics-cert\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.850577 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.850564 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9972ef46-4061-4afd-87ed-255e2f89cb1c-manager-config\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.851192 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.851167 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9972ef46-4061-4afd-87ed-255e2f89cb1c-manager-config\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.853109 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.853090 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9972ef46-4061-4afd-87ed-255e2f89cb1c-metrics-cert\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.853109 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.853102 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9972ef46-4061-4afd-87ed-255e2f89cb1c-cert\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.860709 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.860684 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bz2p\" (UniqueName: \"kubernetes.io/projected/9972ef46-4061-4afd-87ed-255e2f89cb1c-kube-api-access-2bz2p\") pod \"lws-controller-manager-f9f549974-fqwtj\" (UID: \"9972ef46-4061-4afd-87ed-255e2f89cb1c\") " pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:49.988389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:49.988335 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:50.258215 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:50.258145 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj"] Sep 09 21:25:50.261828 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:25:50.261800 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9972ef46_4061_4afd_87ed_255e2f89cb1c.slice/crio-f3d787b48e114a2796b2827f845fc64e8ed354e5021858e272da21365f3d6a62 WatchSource:0}: Error finding container f3d787b48e114a2796b2827f845fc64e8ed354e5021858e272da21365f3d6a62: Status 404 returned error can't find the container with id f3d787b48e114a2796b2827f845fc64e8ed354e5021858e272da21365f3d6a62 Sep 09 21:25:50.341069 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:50.341038 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" event={"ID":"9972ef46-4061-4afd-87ed-255e2f89cb1c","Type":"ContainerStarted","Data":"f3d787b48e114a2796b2827f845fc64e8ed354e5021858e272da21365f3d6a62"} Sep 09 21:25:53.356784 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:53.356745 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" event={"ID":"9972ef46-4061-4afd-87ed-255e2f89cb1c","Type":"ContainerStarted","Data":"7968e5f8522dfc2ec17c858abd0d86050d511012798219a4b42b7f1fbc47140c"} Sep 09 21:25:53.357167 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:53.356929 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:25:53.376821 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:53.376779 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" podStartSLOduration=1.495114968 podStartE2EDuration="4.376765284s" totalImagesPullingTime="2.881650316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:25:53.375653763 +0000 UTC m=+955.888390602" watchObservedRunningTime="2026-09-09 21:25:53.376765284 +0000 UTC m=+955.889502102" Sep 09 21:25:58.789486 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.789443 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh"] Sep 09 21:25:58.793417 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.793394 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:58.795579 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.795555 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 09 21:25:58.795708 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.795559 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 09 21:25:58.795977 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.795956 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 09 21:25:58.796086 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.795992 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-q5kml\"" Sep 09 21:25:58.803999 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.803976 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh"] Sep 09 21:25:58.928734 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.928692 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:58.928962 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.928749 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:58.928962 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.928809 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:58.928962 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.928835 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:58.928962 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.928861 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:58.928962 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.928887 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:58.929226 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.928993 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:58.929226 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.929035 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:58.929226 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:58.929077 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf6nn\" (UniqueName: \"kubernetes.io/projected/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-kube-api-access-wf6nn\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.030441 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.030390 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.030441 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.030455 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.030692 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.030507 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.030692 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.030534 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.030771 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.030686 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf6nn\" (UniqueName: \"kubernetes.io/projected/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-kube-api-access-wf6nn\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.030771 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.030745 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.030954 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.030794 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.030954 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.030838 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.030954 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.030912 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.030954 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.030923 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.031159 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.030973 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.031159 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.031133 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.031259 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.031223 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.031537 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.031514 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.033389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.033356 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.033389 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.033383 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.039610 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.039536 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.039731 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.039671 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf6nn\" (UniqueName: \"kubernetes.io/projected/cf3631eb-3f3b-4b42-8a6a-0da0ff683f85-kube-api-access-wf6nn\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh\" (UID: \"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.105509 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.105465 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:25:59.383845 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:25:59.383816 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh"] Sep 09 21:25:59.387665 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:25:59.387639 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3631eb_3f3b_4b42_8a6a_0da0ff683f85.slice/crio-98328f069c35a1ae316112773ab5d4f3b99bc691609d47fd536319779d54560e WatchSource:0}: Error finding container 98328f069c35a1ae316112773ab5d4f3b99bc691609d47fd536319779d54560e: Status 404 returned error can't find the container with id 98328f069c35a1ae316112773ab5d4f3b99bc691609d47fd536319779d54560e Sep 09 21:26:00.388566 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:00.388524 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" event={"ID":"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85","Type":"ContainerStarted","Data":"98328f069c35a1ae316112773ab5d4f3b99bc691609d47fd536319779d54560e"} Sep 09 21:26:02.180552 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.180510 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 09 21:26:02.180956 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.180609 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 09 21:26:02.180956 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.180655 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 09 21:26:02.395584 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.395546 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wxnpq"] Sep 09 21:26:02.398459 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.398436 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" Sep 09 21:26:02.399664 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.399631 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" event={"ID":"cf3631eb-3f3b-4b42-8a6a-0da0ff683f85","Type":"ContainerStarted","Data":"3fceed67383e0bad94998e4c6255db2021076178bdb2eb62b20c0f598ecc2f4d"} Sep 09 21:26:02.400716 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.400699 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 09 21:26:02.400829 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.400811 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9rfjl\"" Sep 09 21:26:02.407279 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.407227 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wxnpq"] Sep 09 21:26:02.436066 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.435973 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" podStartSLOduration=1.6456337840000002 podStartE2EDuration="4.435952568s" totalImagesPullingTime="2.790318784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:26:02.434848644 +0000 UTC m=+964.947585488" watchObservedRunningTime="2026-09-09 21:26:02.435952568 +0000 UTC m=+964.948689390" Sep 09 21:26:02.561696 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.561635 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3338334-a9f4-420e-b8bb-ada0589304d8-cert\") pod \"odh-model-controller-7998c67dc4-wxnpq\" (UID: \"d3338334-a9f4-420e-b8bb-ada0589304d8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" Sep 09 21:26:02.561696 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.561694 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrb7q\" (UniqueName: \"kubernetes.io/projected/d3338334-a9f4-420e-b8bb-ada0589304d8-kube-api-access-mrb7q\") pod \"odh-model-controller-7998c67dc4-wxnpq\" (UID: \"d3338334-a9f4-420e-b8bb-ada0589304d8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" Sep 09 21:26:02.662753 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.662716 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3338334-a9f4-420e-b8bb-ada0589304d8-cert\") pod \"odh-model-controller-7998c67dc4-wxnpq\" (UID: \"d3338334-a9f4-420e-b8bb-ada0589304d8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" Sep 09 21:26:02.662753 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.662769 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrb7q\" (UniqueName: \"kubernetes.io/projected/d3338334-a9f4-420e-b8bb-ada0589304d8-kube-api-access-mrb7q\") pod \"odh-model-controller-7998c67dc4-wxnpq\" (UID: \"d3338334-a9f4-420e-b8bb-ada0589304d8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" Sep 09 21:26:02.663016 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:26:02.662894 2550 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 09 21:26:02.663016 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:26:02.662977 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3338334-a9f4-420e-b8bb-ada0589304d8-cert podName:d3338334-a9f4-420e-b8bb-ada0589304d8 nodeName:}" failed. No retries permitted until 2026-09-09 21:26:03.162951964 +0000 UTC m=+965.675688785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d3338334-a9f4-420e-b8bb-ada0589304d8-cert") pod "odh-model-controller-7998c67dc4-wxnpq" (UID: "d3338334-a9f4-420e-b8bb-ada0589304d8") : secret "odh-model-controller-webhook-cert" not found Sep 09 21:26:02.673414 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:02.673387 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrb7q\" (UniqueName: \"kubernetes.io/projected/d3338334-a9f4-420e-b8bb-ada0589304d8-kube-api-access-mrb7q\") pod \"odh-model-controller-7998c67dc4-wxnpq\" (UID: \"d3338334-a9f4-420e-b8bb-ada0589304d8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" Sep 09 21:26:03.000709 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.000677 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xttvl"] Sep 09 21:26:03.003298 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.003280 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" Sep 09 21:26:03.005633 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.005612 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 09 21:26:03.016126 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.016102 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xttvl"] Sep 09 21:26:03.106125 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.106098 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:26:03.107414 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.107391 2550 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Sep 09 21:26:03.107545 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.107434 2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" podUID="cf3631eb-3f3b-4b42-8a6a-0da0ff683f85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Sep 09 21:26:03.168633 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.168609 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plzk4\" (UniqueName: \"kubernetes.io/projected/c1d9a0d1-53c9-4fbd-b988-e6b75abab53c-kube-api-access-plzk4\") pod \"model-serving-api-54d89bd5fc-xttvl\" (UID: \"c1d9a0d1-53c9-4fbd-b988-e6b75abab53c\") " pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" Sep 09 21:26:03.168782 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.168747 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3338334-a9f4-420e-b8bb-ada0589304d8-cert\") pod \"odh-model-controller-7998c67dc4-wxnpq\" (UID: \"d3338334-a9f4-420e-b8bb-ada0589304d8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" Sep 09 21:26:03.168884 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.168865 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1d9a0d1-53c9-4fbd-b988-e6b75abab53c-tls-certs\") pod \"model-serving-api-54d89bd5fc-xttvl\" (UID: \"c1d9a0d1-53c9-4fbd-b988-e6b75abab53c\") " pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" Sep 09 21:26:03.171344 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.171308 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3338334-a9f4-420e-b8bb-ada0589304d8-cert\") pod \"odh-model-controller-7998c67dc4-wxnpq\" (UID: \"d3338334-a9f4-420e-b8bb-ada0589304d8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" Sep 09 21:26:03.270210 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.270123 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plzk4\" (UniqueName: \"kubernetes.io/projected/c1d9a0d1-53c9-4fbd-b988-e6b75abab53c-kube-api-access-plzk4\") pod \"model-serving-api-54d89bd5fc-xttvl\" (UID: \"c1d9a0d1-53c9-4fbd-b988-e6b75abab53c\") " pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" Sep 09 21:26:03.270597 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.270255 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1d9a0d1-53c9-4fbd-b988-e6b75abab53c-tls-certs\") pod \"model-serving-api-54d89bd5fc-xttvl\" (UID: \"c1d9a0d1-53c9-4fbd-b988-e6b75abab53c\") " pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" Sep 09 21:26:03.272971 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.272944 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1d9a0d1-53c9-4fbd-b988-e6b75abab53c-tls-certs\") pod \"model-serving-api-54d89bd5fc-xttvl\" (UID: \"c1d9a0d1-53c9-4fbd-b988-e6b75abab53c\") " pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" Sep 09 21:26:03.278680 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.278654 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plzk4\" (UniqueName: \"kubernetes.io/projected/c1d9a0d1-53c9-4fbd-b988-e6b75abab53c-kube-api-access-plzk4\") pod \"model-serving-api-54d89bd5fc-xttvl\" (UID: \"c1d9a0d1-53c9-4fbd-b988-e6b75abab53c\") " pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" Sep 09 21:26:03.311451 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.311421 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" Sep 09 21:26:03.317687 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.317666 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" Sep 09 21:26:03.590781 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.590651 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wxnpq"] Sep 09 21:26:03.593493 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:26:03.593465 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3338334_a9f4_420e_b8bb_ada0589304d8.slice/crio-ac8cbf23791c3116b9655f336dfc1dff686867f78cb0526687c75b7daef32129 WatchSource:0}: Error finding container ac8cbf23791c3116b9655f336dfc1dff686867f78cb0526687c75b7daef32129: Status 404 returned error can't find the container with id ac8cbf23791c3116b9655f336dfc1dff686867f78cb0526687c75b7daef32129 Sep 09 21:26:03.599708 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:03.599669 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xttvl"] Sep 09 21:26:03.601985 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:26:03.601951 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d9a0d1_53c9_4fbd_b988_e6b75abab53c.slice/crio-fb8ffbaebaffcfd2e32bb8ed4c4e25c65e2370b6fb259a61bc9d5d3f74fafd14 WatchSource:0}: Error finding container fb8ffbaebaffcfd2e32bb8ed4c4e25c65e2370b6fb259a61bc9d5d3f74fafd14: Status 404 returned error can't find the container with id fb8ffbaebaffcfd2e32bb8ed4c4e25c65e2370b6fb259a61bc9d5d3f74fafd14 Sep 09 21:26:04.109664 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:04.109637 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:26:04.363825 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:04.363756 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-f9f549974-fqwtj" Sep 09 21:26:04.426500 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:04.426415 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" event={"ID":"c1d9a0d1-53c9-4fbd-b988-e6b75abab53c","Type":"ContainerStarted","Data":"fb8ffbaebaffcfd2e32bb8ed4c4e25c65e2370b6fb259a61bc9d5d3f74fafd14"} Sep 09 21:26:04.429336 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:04.429220 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" event={"ID":"d3338334-a9f4-420e-b8bb-ada0589304d8","Type":"ContainerStarted","Data":"ac8cbf23791c3116b9655f336dfc1dff686867f78cb0526687c75b7daef32129"} Sep 09 21:26:04.429336 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:04.429272 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:26:04.430682 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:04.430491 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh" Sep 09 21:26:08.447929 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:08.447893 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" event={"ID":"c1d9a0d1-53c9-4fbd-b988-e6b75abab53c","Type":"ContainerStarted","Data":"c447eae54572189ed6be8a595e07f1e37bceea50a7acd18395f1579137f510cb"} Sep 09 21:26:08.448441 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:08.448054 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" Sep 09 21:26:08.449591 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:08.449567 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" event={"ID":"d3338334-a9f4-420e-b8bb-ada0589304d8","Type":"ContainerStarted","Data":"a5376477fc6ebdad49e37bed5f5662ea6ceb6d70ae55d326270454a8c1564791"} Sep 09 21:26:08.449721 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:08.449707 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" Sep 09 21:26:08.464425 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:08.464383 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" podStartSLOduration=2.448756729 podStartE2EDuration="6.464370364s" totalImagesPullingTime="4.015613635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:26:08.463788543 +0000 UTC m=+970.976525372" watchObservedRunningTime="2026-09-09 21:26:08.464370364 +0000 UTC m=+970.977107184" Sep 09 21:26:08.479408 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:08.479367 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" podStartSLOduration=2.444819157 podStartE2EDuration="6.47935167s" totalImagesPullingTime="4.034532513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:26:08.479032237 +0000 UTC m=+970.991769082" watchObservedRunningTime="2026-09-09 21:26:08.47935167 +0000 UTC m=+970.992088492" Sep 09 21:26:10.316895 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.316860 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4"] Sep 09 21:26:10.319103 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.319087 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.321608 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.321587 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 09 21:26:10.321715 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.321590 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 09 21:26:10.321715 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.321687 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zwxc2\"" Sep 09 21:26:10.321827 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.321811 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 09 21:26:10.327106 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.327086 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.327182 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.327123 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.327182 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.327141 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.327258 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.327223 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.327300 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.327255 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.327414 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.327383 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.327463 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.327440 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8x56\" (UniqueName: \"kubernetes.io/projected/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-kube-api-access-v8x56\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.346527 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.346506 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4"] Sep 09 21:26:10.428622 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.428591 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.428768 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.428630 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.428768 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.428679 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.428768 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.428705 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.428934 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.428775 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.428934 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.428807 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8x56\" (UniqueName: \"kubernetes.io/projected/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-kube-api-access-v8x56\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.428934 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.428848 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.429859 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.429829 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.431514 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.431469 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.431610 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.431511 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.431724 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.431680 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.431817 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.431800 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.438828 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.438803 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.439611 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.439572 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8x56\" (UniqueName: \"kubernetes.io/projected/6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce-kube-api-access-v8x56\") pod \"istiod-openshift-gateway-5677ccb49b-z55l4\" (UID: \"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.628714 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.628696 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:10.922365 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:10.922342 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4"] Sep 09 21:26:10.924774 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:26:10.924745 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b2c5dd0_cf99_4cf6_a875_be0dfdd418ce.slice/crio-dc73819616331b5a1e6b37d5b23e4606529db6e3dacc7253e6bce571233a7867 WatchSource:0}: Error finding container dc73819616331b5a1e6b37d5b23e4606529db6e3dacc7253e6bce571233a7867: Status 404 returned error can't find the container with id dc73819616331b5a1e6b37d5b23e4606529db6e3dacc7253e6bce571233a7867 Sep 09 21:26:11.465069 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:11.465035 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" event={"ID":"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce","Type":"ContainerStarted","Data":"dc73819616331b5a1e6b37d5b23e4606529db6e3dacc7253e6bce571233a7867"} Sep 09 21:26:13.491587 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:13.491543 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 09 21:26:13.491927 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:13.491624 2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 09 21:26:14.481550 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:14.481513 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" event={"ID":"6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce","Type":"ContainerStarted","Data":"49df18e4961b9952d3a4c74bbfdfcee459f06e7ee62bb8c80508c72061ba0dac"} Sep 09 21:26:14.481730 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:14.481679 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:14.483660 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:14.483634 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" Sep 09 21:26:14.506485 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:14.506431 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-z55l4" podStartSLOduration=1.942159848 podStartE2EDuration="4.506414133s" totalImagesPullingTime="2.564254285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:26:14.505836909 +0000 UTC m=+977.018573728" watchObservedRunningTime="2026-09-09 21:26:14.506414133 +0000 UTC m=+977.019150952" Sep 09 21:26:19.457501 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:19.457414 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-wxnpq" Sep 09 21:26:19.459311 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:26:19.459284 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-xttvl" Sep 09 21:30:00.166527 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:00.166489 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816490-rkvhm"] Sep 09 21:30:00.169905 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:00.169891 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816490-rkvhm" Sep 09 21:30:00.172399 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:00.172374 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816490-rkvhm"] Sep 09 21:30:00.173364 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:00.173343 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjxtb\"" Sep 09 21:30:00.180469 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:00.180455 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816490-rkvhm" Sep 09 21:30:00.448799 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:00.448775 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816490-rkvhm"] Sep 09 21:30:00.450678 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:30:00.450653 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4360f95b_2484_4f11_aff8_41c1624e0638.slice/crio-1c392d318bc6e6a88d3cccdd359901a30adb303441e6e2f59d82ff2a8fc33ec8 WatchSource:0}: Error finding container 1c392d318bc6e6a88d3cccdd359901a30adb303441e6e2f59d82ff2a8fc33ec8: Status 404 returned error can't find the container with id 1c392d318bc6e6a88d3cccdd359901a30adb303441e6e2f59d82ff2a8fc33ec8 Sep 09 21:30:01.446363 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:01.446309 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816490-rkvhm" event={"ID":"4360f95b-2484-4f11-aff8-41c1624e0638","Type":"ContainerStarted","Data":"1c392d318bc6e6a88d3cccdd359901a30adb303441e6e2f59d82ff2a8fc33ec8"} Sep 09 21:30:03.457091 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:03.457055 2550 generic.go:356] "Generic (PLEG): container finished" podID="4360f95b-2484-4f11-aff8-41c1624e0638" containerID="1dfc283deb59d3e23327c43d3493eb98b945172df0ce9b01ab3d5acd2f1ee1ed" exitCode=7 Sep 09 21:30:03.457525 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:03.457126 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816490-rkvhm" event={"ID":"4360f95b-2484-4f11-aff8-41c1624e0638","Type":"ContainerDied","Data":"1dfc283deb59d3e23327c43d3493eb98b945172df0ce9b01ab3d5acd2f1ee1ed"} Sep 09 21:30:03.457525 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:03.457298 2550 scope.go:117] "RemoveContainer" containerID="1dfc283deb59d3e23327c43d3493eb98b945172df0ce9b01ab3d5acd2f1ee1ed" Sep 09 21:30:04.463244 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:04.463218 2550 generic.go:356] "Generic (PLEG): container finished" podID="4360f95b-2484-4f11-aff8-41c1624e0638" containerID="d903194e0fb912faf1b032eb79e688a3cd6c1a1f17ac3eef1100372cc89a9bf9" exitCode=0 Sep 09 21:30:04.463635 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:04.463291 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816490-rkvhm" event={"ID":"4360f95b-2484-4f11-aff8-41c1624e0638","Type":"ContainerDied","Data":"d903194e0fb912faf1b032eb79e688a3cd6c1a1f17ac3eef1100372cc89a9bf9"} Sep 09 21:30:04.463635 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:04.463351 2550 scope.go:117] "RemoveContainer" containerID="1dfc283deb59d3e23327c43d3493eb98b945172df0ce9b01ab3d5acd2f1ee1ed" Sep 09 21:30:05.587750 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:05.587727 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816490-rkvhm" Sep 09 21:30:06.474949 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:06.474922 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816490-rkvhm" Sep 09 21:30:06.475097 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:06.474927 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816490-rkvhm" event={"ID":"4360f95b-2484-4f11-aff8-41c1624e0638","Type":"ContainerDied","Data":"1c392d318bc6e6a88d3cccdd359901a30adb303441e6e2f59d82ff2a8fc33ec8"} Sep 09 21:30:06.475097 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:06.475072 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c392d318bc6e6a88d3cccdd359901a30adb303441e6e2f59d82ff2a8fc33ec8" Sep 09 21:30:09.990867 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.990841 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg"] Sep 09 21:30:09.991227 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.991127 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4360f95b-2484-4f11-aff8-41c1624e0638" containerName="cleanup" Sep 09 21:30:09.991227 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.991137 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4360f95b-2484-4f11-aff8-41c1624e0638" containerName="cleanup" Sep 09 21:30:09.991227 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.991157 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4360f95b-2484-4f11-aff8-41c1624e0638" containerName="cleanup" Sep 09 21:30:09.991227 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.991163 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4360f95b-2484-4f11-aff8-41c1624e0638" containerName="cleanup" Sep 09 21:30:09.991227 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.991212 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4360f95b-2484-4f11-aff8-41c1624e0638" containerName="cleanup" Sep 09 21:30:09.991227 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.991221 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4360f95b-2484-4f11-aff8-41c1624e0638" containerName="cleanup" Sep 09 21:30:09.994181 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.994165 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:09.996620 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.996598 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 09 21:30:09.997270 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.997246 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 09 21:30:09.997395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.997249 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 09 21:30:09.997395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:09.997288 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6rwsm\"" Sep 09 21:30:10.005337 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.005302 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg"] Sep 09 21:30:10.074932 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.074909 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.075058 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.074939 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7ljl\" (UniqueName: \"kubernetes.io/projected/232fb8ac-e61a-40ec-be08-fd8b80995127-kube-api-access-j7ljl\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.075058 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.074963 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.075058 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.075034 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/232fb8ac-e61a-40ec-be08-fd8b80995127-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.075180 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.075086 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.075180 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.075111 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.175931 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.175903 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/232fb8ac-e61a-40ec-be08-fd8b80995127-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.176061 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.175961 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.176061 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.176003 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.176061 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.176042 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.176222 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.176065 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7ljl\" (UniqueName: \"kubernetes.io/projected/232fb8ac-e61a-40ec-be08-fd8b80995127-kube-api-access-j7ljl\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.176222 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.176194 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.176446 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.176424 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.176446 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.176437 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.176569 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.176534 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.178388 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.178366 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/232fb8ac-e61a-40ec-be08-fd8b80995127-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.178516 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.178499 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/232fb8ac-e61a-40ec-be08-fd8b80995127-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.184376 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.184358 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7ljl\" (UniqueName: \"kubernetes.io/projected/232fb8ac-e61a-40ec-be08-fd8b80995127-kube-api-access-j7ljl\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7d2mg\" (UID: \"232fb8ac-e61a-40ec-be08-fd8b80995127\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.304854 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.304790 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:10.580111 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:10.580086 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg"] Sep 09 21:30:10.582341 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:30:10.582288 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod232fb8ac_e61a_40ec_be08_fd8b80995127.slice/crio-bdeee9dae0b13d6a4c0b7c2169b6702d05aee605e3167be025920fe6f073999d WatchSource:0}: Error finding container bdeee9dae0b13d6a4c0b7c2169b6702d05aee605e3167be025920fe6f073999d: Status 404 returned error can't find the container with id bdeee9dae0b13d6a4c0b7c2169b6702d05aee605e3167be025920fe6f073999d Sep 09 21:30:11.500011 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:11.499965 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" event={"ID":"232fb8ac-e61a-40ec-be08-fd8b80995127","Type":"ContainerStarted","Data":"bdeee9dae0b13d6a4c0b7c2169b6702d05aee605e3167be025920fe6f073999d"} Sep 09 21:30:12.505988 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:12.505952 2550 generic.go:356] "Generic (PLEG): container finished" podID="232fb8ac-e61a-40ec-be08-fd8b80995127" containerID="8bfeb31109dc7d1fc2d112d983a0ae7234341a7358188b6b92293cfd8b23fea5" exitCode=0 Sep 09 21:30:12.506455 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:12.506039 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" event={"ID":"232fb8ac-e61a-40ec-be08-fd8b80995127","Type":"ContainerDied","Data":"8bfeb31109dc7d1fc2d112d983a0ae7234341a7358188b6b92293cfd8b23fea5"} Sep 09 21:30:12.506455 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:12.506213 2550 scope.go:117] "RemoveContainer" containerID="8bfeb31109dc7d1fc2d112d983a0ae7234341a7358188b6b92293cfd8b23fea5" Sep 09 21:30:13.512770 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:13.512740 2550 generic.go:356] "Generic (PLEG): container finished" podID="232fb8ac-e61a-40ec-be08-fd8b80995127" containerID="84e8f72e25b1234161f873c72e4f5a202b91f95d9f9617d0efedff6b31912935" exitCode=0 Sep 09 21:30:13.513197 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:13.512825 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" event={"ID":"232fb8ac-e61a-40ec-be08-fd8b80995127","Type":"ContainerDied","Data":"84e8f72e25b1234161f873c72e4f5a202b91f95d9f9617d0efedff6b31912935"} Sep 09 21:30:13.513197 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:13.512894 2550 scope.go:117] "RemoveContainer" containerID="8bfeb31109dc7d1fc2d112d983a0ae7234341a7358188b6b92293cfd8b23fea5" Sep 09 21:30:13.513197 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:13.513097 2550 scope.go:117] "RemoveContainer" containerID="84e8f72e25b1234161f873c72e4f5a202b91f95d9f9617d0efedff6b31912935" Sep 09 21:30:13.513304 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:30:13.513289 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:30:14.520447 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:14.520425 2550 scope.go:117] "RemoveContainer" containerID="84e8f72e25b1234161f873c72e4f5a202b91f95d9f9617d0efedff6b31912935" Sep 09 21:30:14.520808 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:30:14.520568 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:30:20.305562 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:20.305530 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:20.305936 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:20.305585 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:20.305936 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:20.305603 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:20.305936 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:20.305616 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:20.306043 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:20.305986 2550 scope.go:117] "RemoveContainer" containerID="84e8f72e25b1234161f873c72e4f5a202b91f95d9f9617d0efedff6b31912935" Sep 09 21:30:20.306183 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:30:20.306162 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:30:34.984887 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:34.984859 2550 scope.go:117] "RemoveContainer" containerID="84e8f72e25b1234161f873c72e4f5a202b91f95d9f9617d0efedff6b31912935" Sep 09 21:30:34.987499 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:34.985122 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:30:35.607215 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:35.607133 2550 generic.go:356] "Generic (PLEG): container finished" podID="232fb8ac-e61a-40ec-be08-fd8b80995127" containerID="0864cea25741d98957f5aa3f636ec07650fb4c82f93435a2cf6d30df20fc01bc" exitCode=0 Sep 09 21:30:35.607215 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:35.607181 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" event={"ID":"232fb8ac-e61a-40ec-be08-fd8b80995127","Type":"ContainerDied","Data":"0864cea25741d98957f5aa3f636ec07650fb4c82f93435a2cf6d30df20fc01bc"} Sep 09 21:30:35.607464 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:35.607228 2550 scope.go:117] "RemoveContainer" containerID="84e8f72e25b1234161f873c72e4f5a202b91f95d9f9617d0efedff6b31912935" Sep 09 21:30:35.607611 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:35.607592 2550 scope.go:117] "RemoveContainer" containerID="0864cea25741d98957f5aa3f636ec07650fb4c82f93435a2cf6d30df20fc01bc" Sep 09 21:30:35.607807 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:30:35.607787 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:30:40.305682 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:40.305649 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:40.306102 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:40.305701 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:40.306102 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:40.305719 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:40.306102 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:40.305731 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:30:40.306102 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:40.305989 2550 scope.go:117] "RemoveContainer" containerID="0864cea25741d98957f5aa3f636ec07650fb4c82f93435a2cf6d30df20fc01bc" Sep 09 21:30:40.306267 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:30:40.306153 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:30:50.984966 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:50.984895 2550 scope.go:117] "RemoveContainer" containerID="0864cea25741d98957f5aa3f636ec07650fb4c82f93435a2cf6d30df20fc01bc" Sep 09 21:30:50.985298 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:30:50.985078 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:30:52.697215 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.697181 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc"] Sep 09 21:30:52.705595 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.705570 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.707925 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.707904 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 09 21:30:52.712407 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.712078 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc"] Sep 09 21:30:52.804182 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.804152 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.804182 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.804180 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0eca777-8a80-41df-bd18-14b6391d9c11-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.804343 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.804198 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.804343 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.804220 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhvrg\" (UniqueName: \"kubernetes.io/projected/f0eca777-8a80-41df-bd18-14b6391d9c11-kube-api-access-zhvrg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.804343 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.804300 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.804438 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.804363 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.905101 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.905071 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.905224 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.905103 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0eca777-8a80-41df-bd18-14b6391d9c11-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.905224 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.905122 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.905224 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.905160 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhvrg\" (UniqueName: \"kubernetes.io/projected/f0eca777-8a80-41df-bd18-14b6391d9c11-kube-api-access-zhvrg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.905224 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.905209 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.905497 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.905264 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.905497 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.905488 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.905589 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.905548 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.905632 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.905591 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.907486 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.907463 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0eca777-8a80-41df-bd18-14b6391d9c11-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.907776 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.907758 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0eca777-8a80-41df-bd18-14b6391d9c11-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:52.913796 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:52.913767 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhvrg\" (UniqueName: \"kubernetes.io/projected/f0eca777-8a80-41df-bd18-14b6391d9c11-kube-api-access-zhvrg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc\" (UID: \"f0eca777-8a80-41df-bd18-14b6391d9c11\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:53.015809 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:53.015757 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:30:53.287553 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:53.287527 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc"] Sep 09 21:30:53.289559 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:30:53.289535 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0eca777_8a80_41df_bd18_14b6391d9c11.slice/crio-82d4599fd8d30282c1b39dc7d6c93065aec702c1eac564029c5018c78f99e5f7 WatchSource:0}: Error finding container 82d4599fd8d30282c1b39dc7d6c93065aec702c1eac564029c5018c78f99e5f7: Status 404 returned error can't find the container with id 82d4599fd8d30282c1b39dc7d6c93065aec702c1eac564029c5018c78f99e5f7 Sep 09 21:30:53.690597 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:53.690562 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" event={"ID":"f0eca777-8a80-41df-bd18-14b6391d9c11","Type":"ContainerStarted","Data":"df6e27ddeff40a5a935f214a1e93a2816b667bcc9cc7ff8dba9b734577d1a089"} Sep 09 21:30:53.690736 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:53.690622 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" event={"ID":"f0eca777-8a80-41df-bd18-14b6391d9c11","Type":"ContainerStarted","Data":"82d4599fd8d30282c1b39dc7d6c93065aec702c1eac564029c5018c78f99e5f7"} Sep 09 21:30:53.709580 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:30:53.709539 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" podStartSLOduration=1.534791068 podStartE2EDuration="1.709523963s" totalImagesPullingTime="174.732895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:30:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:30:53.707414626 +0000 UTC m=+1256.220151465" watchObservedRunningTime="2026-09-09 21:30:53.709523963 +0000 UTC m=+1256.222260780" Sep 09 21:31:03.016379 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:03.016346 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:31:03.016762 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:03.016399 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:31:03.028074 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:03.028047 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:31:03.744890 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:03.744823 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc" Sep 09 21:31:05.985040 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:05.985013 2550 scope.go:117] "RemoveContainer" containerID="0864cea25741d98957f5aa3f636ec07650fb4c82f93435a2cf6d30df20fc01bc" Sep 09 21:31:06.748788 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:06.748761 2550 generic.go:356] "Generic (PLEG): container finished" podID="232fb8ac-e61a-40ec-be08-fd8b80995127" containerID="df782762dc71f45ebe98eb46a1da377c3b9a48ee41fe2205c16d39cb2574cbab" exitCode=0 Sep 09 21:31:06.748932 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:06.748794 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" event={"ID":"232fb8ac-e61a-40ec-be08-fd8b80995127","Type":"ContainerDied","Data":"df782762dc71f45ebe98eb46a1da377c3b9a48ee41fe2205c16d39cb2574cbab"} Sep 09 21:31:06.748932 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:06.748839 2550 scope.go:117] "RemoveContainer" containerID="0864cea25741d98957f5aa3f636ec07650fb4c82f93435a2cf6d30df20fc01bc" Sep 09 21:31:06.749164 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:06.749145 2550 scope.go:117] "RemoveContainer" containerID="df782762dc71f45ebe98eb46a1da377c3b9a48ee41fe2205c16d39cb2574cbab" Sep 09 21:31:06.749359 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:31:06.749310 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:31:10.305547 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:10.305511 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:31:10.305906 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:10.305568 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:31:10.305906 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:10.305585 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:31:10.305906 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:10.305599 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:31:10.306023 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:10.305914 2550 scope.go:117] "RemoveContainer" containerID="df782762dc71f45ebe98eb46a1da377c3b9a48ee41fe2205c16d39cb2574cbab" Sep 09 21:31:10.306090 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:31:10.306070 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:31:22.984546 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:22.984519 2550 scope.go:117] "RemoveContainer" containerID="df782762dc71f45ebe98eb46a1da377c3b9a48ee41fe2205c16d39cb2574cbab" Sep 09 21:31:22.984903 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:31:22.984673 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:31:34.985455 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:34.985422 2550 scope.go:117] "RemoveContainer" containerID="df782762dc71f45ebe98eb46a1da377c3b9a48ee41fe2205c16d39cb2574cbab" Sep 09 21:31:34.985996 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:31:34.985640 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:31:48.985102 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:48.985075 2550 scope.go:117] "RemoveContainer" containerID="df782762dc71f45ebe98eb46a1da377c3b9a48ee41fe2205c16d39cb2574cbab" Sep 09 21:31:49.935651 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:49.935619 2550 generic.go:356] "Generic (PLEG): container finished" podID="232fb8ac-e61a-40ec-be08-fd8b80995127" containerID="e4fa22b754216e704eb470732151a4c9fdb42304969a997cc75069010f68ff1f" exitCode=0 Sep 09 21:31:49.935829 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:49.935698 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" event={"ID":"232fb8ac-e61a-40ec-be08-fd8b80995127","Type":"ContainerDied","Data":"e4fa22b754216e704eb470732151a4c9fdb42304969a997cc75069010f68ff1f"} Sep 09 21:31:49.935829 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:49.935764 2550 scope.go:117] "RemoveContainer" containerID="df782762dc71f45ebe98eb46a1da377c3b9a48ee41fe2205c16d39cb2574cbab" Sep 09 21:31:49.936178 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:49.936164 2550 scope.go:117] "RemoveContainer" containerID="e4fa22b754216e704eb470732151a4c9fdb42304969a997cc75069010f68ff1f" Sep 09 21:31:49.936425 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:31:49.936395 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:31:50.305887 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:50.305793 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:31:50.305887 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:50.305862 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:31:50.305887 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:50.305878 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:31:50.305887 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:50.305893 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:31:50.941828 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:31:50.941804 2550 scope.go:117] "RemoveContainer" containerID="e4fa22b754216e704eb470732151a4c9fdb42304969a997cc75069010f68ff1f" Sep 09 21:31:50.942000 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:31:50.941971 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:32:03.984845 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:32:03.984811 2550 scope.go:117] "RemoveContainer" containerID="e4fa22b754216e704eb470732151a4c9fdb42304969a997cc75069010f68ff1f" Sep 09 21:32:03.985304 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:32:03.985043 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:32:17.987399 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:32:17.987368 2550 scope.go:117] "RemoveContainer" containerID="e4fa22b754216e704eb470732151a4c9fdb42304969a997cc75069010f68ff1f" Sep 09 21:32:17.987982 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:32:17.987603 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:32:30.984998 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:32:30.984946 2550 scope.go:117] "RemoveContainer" containerID="e4fa22b754216e704eb470732151a4c9fdb42304969a997cc75069010f68ff1f" Sep 09 21:32:30.985460 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:32:30.985151 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:32:44.984830 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:32:44.984787 2550 scope.go:117] "RemoveContainer" containerID="e4fa22b754216e704eb470732151a4c9fdb42304969a997cc75069010f68ff1f" Sep 09 21:32:44.985289 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:32:44.985036 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:32:58.985272 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:32:58.985238 2550 scope.go:117] "RemoveContainer" containerID="e4fa22b754216e704eb470732151a4c9fdb42304969a997cc75069010f68ff1f" Sep 09 21:32:58.985778 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:32:58.985464 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:33:10.985583 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:10.985549 2550 scope.go:117] "RemoveContainer" containerID="e4fa22b754216e704eb470732151a4c9fdb42304969a997cc75069010f68ff1f" Sep 09 21:33:11.308970 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:11.308932 2550 generic.go:356] "Generic (PLEG): container finished" podID="232fb8ac-e61a-40ec-be08-fd8b80995127" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" exitCode=0 Sep 09 21:33:11.308970 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:11.308973 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" event={"ID":"232fb8ac-e61a-40ec-be08-fd8b80995127","Type":"ContainerDied","Data":"b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78"} Sep 09 21:33:11.309199 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:11.309021 2550 scope.go:117] "RemoveContainer" containerID="e4fa22b754216e704eb470732151a4c9fdb42304969a997cc75069010f68ff1f" Sep 09 21:33:11.309426 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:11.309404 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:33:11.309636 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:33:11.309612 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:33:20.305789 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:20.305752 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:33:20.306173 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:20.305818 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:33:20.306173 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:20.305838 2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:33:20.306173 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:20.305856 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" Sep 09 21:33:20.306277 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:20.306233 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:33:20.306451 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:33:20.306431 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:33:34.986104 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:34.986052 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:33:34.986712 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:33:34.986414 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:33:48.984737 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:48.984650 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:33:48.985116 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:33:48.984904 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:33:59.984963 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:33:59.984918 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:33:59.985511 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:33:59.985175 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:34:13.985038 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:34:13.984993 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:34:13.985586 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:34:13.985198 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:34:25.985364 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:34:25.985309 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:34:25.985763 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:34:25.985525 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:34:40.985578 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:34:40.985541 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:34:40.986010 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:34:40.985714 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:34:55.984842 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:34:55.984806 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:34:55.985351 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:34:55.984964 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:35:08.985474 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:08.985433 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:35:08.985866 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:35:08.985624 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:35:15.104372 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:15.104344 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xttvl_c1d9a0d1-53c9-4fbd-b988-e6b75abab53c/server/0.log" Sep 09 21:35:15.221892 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:15.221862 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wxnpq_d3338334-a9f4-420e-b8bb-ada0589304d8/manager/0.log" Sep 09 21:35:15.336273 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:15.336247 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66dbc47bfc-nml7v_44e2c50e-81fd-4d6d-95a1-b441ade7cdd9/copy-manifests/0.log" Sep 09 21:35:15.349017 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:15.348980 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66dbc47bfc-nml7v_44e2c50e-81fd-4d6d-95a1-b441ade7cdd9/manager/0.log" Sep 09 21:35:16.074286 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:16.074256 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29816490-rkvhm_4360f95b-2484-4f11-aff8-41c1624e0638/cleanup/1.log" Sep 09 21:35:18.025817 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:18.025786 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh_cf3631eb-3f3b-4b42-8a6a-0da0ff683f85/istio-proxy/0.log" Sep 09 21:35:18.143903 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:18.143872 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-z55l4_6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce/discovery/0.log" Sep 09 21:35:19.189133 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:19.189057 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-859856b874-7d2mg_232fb8ac-e61a-40ec-be08-fd8b80995127/main/5.log" Sep 09 21:35:19.662448 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:19.662421 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fs2vtc_f0eca777-8a80-41df-bd18-14b6391d9c11/main/0.log" Sep 09 21:35:19.985264 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:19.985179 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:35:19.985436 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:35:19.985404 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:35:28.668262 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:28.668223 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kggbk_4211bff4-82e4-4201-b854-de56fd0b7e14/global-pull-secret-syncer/0.log" Sep 09 21:35:28.849361 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:28.849333 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qrjzg_e8885188-4e52-4325-8d27-8f3ab4700578/konnectivity-agent/0.log" Sep 09 21:35:28.869655 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:28.869630 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-60.ec2.internal_d53d5013d65214a919bdfb5b967b3585/haproxy/0.log" Sep 09 21:35:32.984908 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:32.984879 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:35:32.985345 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:35:32.985050 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:35:35.153527 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.153497 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20ed06a5-a29d-48ec-b799-f12bcfabcb82/alertmanager/0.log" Sep 09 21:35:35.181444 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.181423 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20ed06a5-a29d-48ec-b799-f12bcfabcb82/config-reloader/0.log" Sep 09 21:35:35.204756 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.204739 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20ed06a5-a29d-48ec-b799-f12bcfabcb82/kube-rbac-proxy-web/0.log" Sep 09 21:35:35.226332 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.226301 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20ed06a5-a29d-48ec-b799-f12bcfabcb82/kube-rbac-proxy/0.log" Sep 09 21:35:35.250288 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.250272 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20ed06a5-a29d-48ec-b799-f12bcfabcb82/kube-rbac-proxy-metric/0.log" Sep 09 21:35:35.278991 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.278969 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20ed06a5-a29d-48ec-b799-f12bcfabcb82/prom-label-proxy/0.log" Sep 09 21:35:35.303645 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.303626 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20ed06a5-a29d-48ec-b799-f12bcfabcb82/init-config-reloader/0.log" Sep 09 21:35:35.465092 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.465023 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-85b67464c9-cmt7t_8d8bf1d0-188f-471c-beeb-4519940b493f/metrics-server/0.log" Sep 09 21:35:35.533163 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.533143 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gkwg5_1a65e9c3-a61b-4ce5-baba-4b101f963a66/node-exporter/0.log" Sep 09 21:35:35.558827 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.558806 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gkwg5_1a65e9c3-a61b-4ce5-baba-4b101f963a66/kube-rbac-proxy/0.log" Sep 09 21:35:35.581515 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.581498 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gkwg5_1a65e9c3-a61b-4ce5-baba-4b101f963a66/init-textfile/0.log" Sep 09 21:35:35.818939 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.818885 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-z4chv_ec6e14a4-77f8-4698-b2e7-7c6fada680d8/kube-rbac-proxy-main/0.log" Sep 09 21:35:35.841394 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.841370 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-z4chv_ec6e14a4-77f8-4698-b2e7-7c6fada680d8/kube-rbac-proxy-self/0.log" Sep 09 21:35:35.864851 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:35.864819 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-z4chv_ec6e14a4-77f8-4698-b2e7-7c6fada680d8/openshift-state-metrics/0.log" Sep 09 21:35:38.918652 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:38.918628 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-865cfd664b-8pjwv_1ce39040-5a9b-47a5-a433-2a33ad972b99/console/0.log" Sep 09 21:35:39.884384 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:39.884354 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj"] Sep 09 21:35:39.887779 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:39.887758 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:39.890058 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:39.890039 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-znttn\"/\"openshift-service-ca.crt\"" Sep 09 21:35:39.890152 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:39.890088 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-znttn\"/\"kube-root-ca.crt\"" Sep 09 21:35:39.890713 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:39.890698 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-znttn\"/\"default-dockercfg-96mk2\"" Sep 09 21:35:39.897921 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:39.897899 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj"] Sep 09 21:35:39.988236 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:39.988217 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-proc\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:39.988638 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:39.988255 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-lib-modules\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:39.988638 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:39.988293 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-sys\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:39.988638 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:39.988312 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vs66\" (UniqueName: \"kubernetes.io/projected/2c26602e-0493-467a-82a6-532346e0687e-kube-api-access-2vs66\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:39.988638 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:39.988358 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-podres\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.088987 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.088958 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-proc\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.089104 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.089022 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-lib-modules\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.089104 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.089069 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-sys\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.089104 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.089077 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-proc\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.089104 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.089093 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vs66\" (UniqueName: \"kubernetes.io/projected/2c26602e-0493-467a-82a6-532346e0687e-kube-api-access-2vs66\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.089267 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.089120 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-sys\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.089267 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.089124 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-podres\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.089267 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.089141 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-lib-modules\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.089267 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.089187 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c26602e-0493-467a-82a6-532346e0687e-podres\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.097310 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.097293 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vs66\" (UniqueName: \"kubernetes.io/projected/2c26602e-0493-467a-82a6-532346e0687e-kube-api-access-2vs66\") pod \"perf-node-gather-daemonset-hnxvj\" (UID: \"2c26602e-0493-467a-82a6-532346e0687e\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.197784 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.197719 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:40.471261 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.471238 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj"] Sep 09 21:35:40.473814 ip-10-0-128-60 kubenswrapper[2550]: W0909 21:35:40.473793 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2c26602e_0493_467a_82a6_532346e0687e.slice/crio-9921f10c607cb646a28f2bdb4bce61e395aadcc001965e50bbcba7c5d26159cf WatchSource:0}: Error finding container 9921f10c607cb646a28f2bdb4bce61e395aadcc001965e50bbcba7c5d26159cf: Status 404 returned error can't find the container with id 9921f10c607cb646a28f2bdb4bce61e395aadcc001965e50bbcba7c5d26159cf Sep 09 21:35:40.858685 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.858627 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jcd2g_d73b7970-9da3-44fb-bd20-efcc68505662/dns/0.log" Sep 09 21:35:40.887268 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:40.887250 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jcd2g_d73b7970-9da3-44fb-bd20-efcc68505662/kube-rbac-proxy/0.log" Sep 09 21:35:41.002123 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:41.002092 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" event={"ID":"2c26602e-0493-467a-82a6-532346e0687e","Type":"ContainerStarted","Data":"5a5b609d03a89274bb0297d39677e02b1ac45d469f0c5e6de29cc5a409cd0b9d"} Sep 09 21:35:41.002539 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:41.002170 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:41.002539 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:41.002188 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" event={"ID":"2c26602e-0493-467a-82a6-532346e0687e","Type":"ContainerStarted","Data":"9921f10c607cb646a28f2bdb4bce61e395aadcc001965e50bbcba7c5d26159cf"} Sep 09 21:35:41.009032 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:41.009008 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h8h2f_4ab41251-7b22-458e-83c1-4dc1ed95611a/dns-node-resolver/0.log" Sep 09 21:35:41.022334 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:41.022281 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" podStartSLOduration=2.022268497 podStartE2EDuration="2.022268497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:35:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:35:41.021902695 +0000 UTC m=+1543.534639534" watchObservedRunningTime="2026-09-09 21:35:41.022268497 +0000 UTC m=+1543.535005364" Sep 09 21:35:41.616283 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:41.616254 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-d965ffd6c-xsjkg_39b28238-b7ad-42ea-ac40-8fd800000b8e/registry/0.log" Sep 09 21:35:41.662605 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:41.662581 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rx56w_fdd3337d-7102-4a4b-826f-093466b1014d/node-ca/0.log" Sep 09 21:35:42.686452 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:42.686417 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh_cf3631eb-3f3b-4b42-8a6a-0da0ff683f85/istio-proxy/0.log" Sep 09 21:35:42.733187 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:42.733158 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-z55l4_6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce/discovery/0.log" Sep 09 21:35:43.522605 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:43.522573 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jq8z4_3ac31950-bf71-41ae-97ce-a45f4533baf0/serve-healthcheck-canary/0.log" Sep 09 21:35:44.152332 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:44.152302 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jvgd8_78ce6eba-2e3e-4a0e-96ad-13be205f151e/kube-rbac-proxy/0.log" Sep 09 21:35:44.172854 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:44.172829 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jvgd8_78ce6eba-2e3e-4a0e-96ad-13be205f151e/exporter/0.log" Sep 09 21:35:44.196445 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:44.196422 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jvgd8_78ce6eba-2e3e-4a0e-96ad-13be205f151e/extractor/0.log" Sep 09 21:35:46.462508 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:46.462477 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xttvl_c1d9a0d1-53c9-4fbd-b988-e6b75abab53c/server/0.log" Sep 09 21:35:46.511657 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:46.511630 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wxnpq_d3338334-a9f4-420e-b8bb-ada0589304d8/manager/0.log" Sep 09 21:35:46.554128 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:46.554108 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66dbc47bfc-nml7v_44e2c50e-81fd-4d6d-95a1-b441ade7cdd9/manager/0.log" Sep 09 21:35:46.569701 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:46.569681 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66dbc47bfc-nml7v_44e2c50e-81fd-4d6d-95a1-b441ade7cdd9/copy-manifests/0.log" Sep 09 21:35:47.016140 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:47.016119 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-hnxvj" Sep 09 21:35:47.833754 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:47.833726 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-f9f549974-fqwtj_9972ef46-4061-4afd-87ed-255e2f89cb1c/manager/0.log" Sep 09 21:35:47.990958 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:47.990934 2550 scope.go:117] "RemoveContainer" containerID="b02a657698a7d639895ab300424883f85f871d11f2eb111ed6b82b7196edbc78" Sep 09 21:35:47.991116 ip-10-0-128-60 kubenswrapper[2550]: E0909 21:35:47.991098 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7d2mg_llm(232fb8ac-e61a-40ec-be08-fd8b80995127)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7d2mg" podUID="232fb8ac-e61a-40ec-be08-fd8b80995127" Sep 09 21:35:48.509070 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:48.509044 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh_cf3631eb-3f3b-4b42-8a6a-0da0ff683f85/istio-proxy/0.log" Sep 09 21:35:48.544725 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:48.544699 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-z55l4_6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce/discovery/0.log" Sep 09 21:35:49.722793 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:49.722755 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-p42bh_cf3631eb-3f3b-4b42-8a6a-0da0ff683f85/istio-proxy/0.log" Sep 09 21:35:49.758417 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:49.758392 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-z55l4_6b2c5dd0-cf99-4cf6-a875-be0dfdd418ce/discovery/0.log" Sep 09 21:35:53.409695 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:53.409666 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-qvnh4_23f7aac5-fb7d-465c-a414-20b2b4689e25/migrator/0.log" Sep 09 21:35:53.442666 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:53.442643 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-qvnh4_23f7aac5-fb7d-465c-a414-20b2b4689e25/graceful-termination/0.log" Sep 09 21:35:54.870165 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:54.870135 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9lzjh_a329dfed-fd34-425d-8372-fd0db6ac6bd7/kube-multus/0.log" Sep 09 21:35:55.222083 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:55.222053 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vth7c_0a04424e-a63f-45bc-bd8b-53b3281e0982/kube-multus-additional-cni-plugins/0.log" Sep 09 21:35:55.244691 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:55.244667 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vth7c_0a04424e-a63f-45bc-bd8b-53b3281e0982/egress-router-binary-copy/0.log" Sep 09 21:35:55.265997 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:55.265975 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vth7c_0a04424e-a63f-45bc-bd8b-53b3281e0982/cni-plugins/0.log" Sep 09 21:35:55.288488 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:55.288469 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vth7c_0a04424e-a63f-45bc-bd8b-53b3281e0982/bond-cni-plugin/0.log" Sep 09 21:35:55.310705 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:55.310684 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vth7c_0a04424e-a63f-45bc-bd8b-53b3281e0982/routeoverride-cni/0.log" Sep 09 21:35:55.334285 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:55.334266 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vth7c_0a04424e-a63f-45bc-bd8b-53b3281e0982/whereabouts-cni-bincopy/0.log" Sep 09 21:35:55.359170 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:55.359156 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vth7c_0a04424e-a63f-45bc-bd8b-53b3281e0982/whereabouts-cni/0.log" Sep 09 21:35:55.542520 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:55.542450 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9jn8c_4b343c4d-2b12-4093-b6de-c4223e67f2bf/network-metrics-daemon/0.log" Sep 09 21:35:55.565207 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:55.565184 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9jn8c_4b343c4d-2b12-4093-b6de-c4223e67f2bf/kube-rbac-proxy/0.log" Sep 09 21:35:56.593395 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:56.593365 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fr2g6_71354de2-a7c4-4336-9a2f-74cf7755fadb/ovn-controller/0.log" Sep 09 21:35:56.657236 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:56.657215 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fr2g6_71354de2-a7c4-4336-9a2f-74cf7755fadb/ovn-acl-logging/0.log" Sep 09 21:35:56.713604 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:56.713583 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fr2g6_71354de2-a7c4-4336-9a2f-74cf7755fadb/kube-rbac-proxy-node/0.log" Sep 09 21:35:56.741523 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:56.741505 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fr2g6_71354de2-a7c4-4336-9a2f-74cf7755fadb/kube-rbac-proxy-ovn-metrics/0.log" Sep 09 21:35:56.766155 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:56.766138 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fr2g6_71354de2-a7c4-4336-9a2f-74cf7755fadb/northd/0.log" Sep 09 21:35:56.786229 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:56.786209 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fr2g6_71354de2-a7c4-4336-9a2f-74cf7755fadb/nbdb/0.log" Sep 09 21:35:56.807002 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:56.806952 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fr2g6_71354de2-a7c4-4336-9a2f-74cf7755fadb/sbdb/0.log" Sep 09 21:35:56.844710 ip-10-0-128-60 kubenswrapper[2550]: I0909 21:35:56.844691 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fr2g6_71354de2-a7c4-4336-9a2f-74cf7755fadb/ovnkube-controller/0.log"