Sep 17 19:06:30.422069 ip-10-0-135-16 systemd[1]: Starting Kubernetes Kubelet... Sep 17 19:06:30.803934 ip-10-0-135-16 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 19:06:30.803934 ip-10-0-135-16 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 17 19:06:30.803934 ip-10-0-135-16 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 19:06:30.804663 ip-10-0-135-16 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 17 19:06:30.804663 ip-10-0-135-16 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 19:06:30.805877 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.804692 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 17 19:06:30.811342 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811325 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:30.811342 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811343 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811348 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811351 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811355 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811358 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811361 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811364 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811367 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811370 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811374 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811377 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811380 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811383 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811385 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811388 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811391 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811393 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811396 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811399 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811402 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:30.811411 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811405 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811407 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811410 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811413 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811417 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811421 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811424 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811429 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811433 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811435 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811438 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811441 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811444 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811447 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811449 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811452 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811456 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811459 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811462 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:30.811934 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811465 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811468 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811471 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811473 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811476 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811478 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811481 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811485 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811487 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811490 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811492 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811495 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811499 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811501 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811504 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811508 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811511 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811513 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811516 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811518 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:30.812404 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811521 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811524 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811526 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811529 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811531 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811536 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811538 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811541 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811544 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811546 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811549 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811551 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811554 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811556 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811559 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811561 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811564 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811567 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811569 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:30.812925 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811572 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811574 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811577 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811579 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811582 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811585 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811587 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811590 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811593 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811596 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811599 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811601 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811604 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811607 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811609 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811612 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811614 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811617 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811619 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.811623 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:30.813378 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812332 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812338 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812341 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812345 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812348 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812351 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812353 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812356 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812359 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812362 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812365 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812367 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812370 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812373 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812376 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812379 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812381 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812384 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812387 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812389 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:30.813883 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812393 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812395 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812400 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812404 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812408 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812411 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812415 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812418 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812421 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812424 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812426 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812429 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812432 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812435 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812438 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812440 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812443 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812445 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812448 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:30.814410 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812450 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812454 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812457 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812459 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812462 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812464 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812467 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812469 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812472 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812475 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812477 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812480 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812483 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812487 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812489 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812492 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812495 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812498 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812500 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812503 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:30.814887 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812505 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812508 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812510 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812513 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812515 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812518 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812521 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812523 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812526 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812528 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812531 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812533 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812536 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812538 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812541 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812544 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812546 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812549 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812551 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812554 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:30.815392 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812556 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812560 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812564 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812567 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812569 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812574 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812578 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812581 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812584 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812587 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812589 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812592 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812594 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812597 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812600 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812603 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812606 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812608 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812621 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:30.815900 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.812625 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813018 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813023 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813026 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813029 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813033 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813036 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813039 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813041 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813044 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813047 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813050 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813053 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813057 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813060 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813062 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813065 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813068 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813071 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813074 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:30.816361 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813076 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813079 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813082 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813084 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813087 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813090 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813092 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813095 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813098 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813101 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813104 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813106 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813109 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813112 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813114 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813117 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813119 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813122 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813125 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813127 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:30.816879 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813130 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813132 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813137 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813140 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813142 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813145 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813148 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813150 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813153 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813157 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813160 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813163 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813166 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813169 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813172 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813175 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813178 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813180 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813183 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:30.817381 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813186 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813188 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813192 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813195 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813198 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813200 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813203 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813206 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813208 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813211 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813214 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813216 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813219 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813221 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813224 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813226 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813230 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813232 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813235 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813237 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:30.817866 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813240 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813243 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813245 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813248 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813251 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813254 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813256 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813259 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813261 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813264 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813266 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813269 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813271 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813274 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813276 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813279 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813281 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813284 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813286 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813289 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:30.818341 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.813292 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813368 2563 flags.go:64] FLAG: --address="0.0.0.0" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813403 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813410 2563 flags.go:64] FLAG: --anonymous-auth="true" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813415 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813419 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813422 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813427 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813431 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813434 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813437 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813441 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813444 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813448 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813451 2563 flags.go:64] FLAG: --cgroup-root="" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813456 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813459 2563 flags.go:64] FLAG: --client-ca-file="" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813462 2563 flags.go:64] FLAG: --cloud-provider="external" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813465 2563 flags.go:64] FLAG: --cluster-dns="[]" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813469 2563 flags.go:64] FLAG: --cluster-domain="" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813472 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813475 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813479 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813482 2563 flags.go:64] FLAG: --container-log-max-files="5" Sep 17 19:06:30.818836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813486 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813488 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813491 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813494 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813497 2563 flags.go:64] FLAG: --contention-profiling="false" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813500 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813503 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813506 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813510 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813513 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813516 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813519 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813522 2563 flags.go:64] FLAG: --enable-load-reader="false" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813524 2563 flags.go:64] FLAG: --enable-server="true" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813527 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813532 2563 flags.go:64] FLAG: --event-burst="100" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813535 2563 flags.go:64] FLAG: --event-qps="50" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813538 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813541 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813544 2563 flags.go:64] FLAG: --eviction-hard="" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813549 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813552 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813555 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813560 2563 flags.go:64] FLAG: --eviction-soft="" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813563 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 17 19:06:30.819413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813566 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813569 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813572 2563 flags.go:64] FLAG: --experimental-mounter-path="" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813575 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813578 2563 flags.go:64] FLAG: --fail-swap-on="true" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813581 2563 flags.go:64] FLAG: --feature-gates="" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813584 2563 flags.go:64] FLAG: --file-check-frequency="20s" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813588 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813591 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813594 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813597 2563 flags.go:64] FLAG: --healthz-port="10248" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813600 2563 flags.go:64] FLAG: --help="false" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813603 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-135-16.ec2.internal" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813606 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813609 2563 flags.go:64] FLAG: --http-check-frequency="20s" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813612 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813616 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813619 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813622 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813625 2563 flags.go:64] FLAG: --image-service-endpoint="" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813627 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813630 2563 flags.go:64] FLAG: --kube-api-burst="100" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813633 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813636 2563 flags.go:64] FLAG: --kube-api-qps="50" Sep 17 19:06:30.820121 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813639 2563 flags.go:64] FLAG: --kube-reserved="" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813642 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813647 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813650 2563 flags.go:64] FLAG: --kubelet-cgroups="" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813652 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813655 2563 flags.go:64] FLAG: --lock-file="" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813660 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813663 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813666 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813671 2563 flags.go:64] FLAG: --log-json-split-stream="false" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813674 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813677 2563 flags.go:64] FLAG: --log-text-split-stream="false" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813680 2563 flags.go:64] FLAG: --logging-format="text" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813683 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813686 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813689 2563 flags.go:64] FLAG: --manifest-url="" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813692 2563 flags.go:64] FLAG: --manifest-url-header="" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813697 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813699 2563 flags.go:64] FLAG: --max-open-files="1000000" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813703 2563 flags.go:64] FLAG: --max-pods="110" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813706 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813709 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813712 2563 flags.go:64] FLAG: --memory-manager-policy="None" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813729 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813732 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 17 19:06:30.820728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813735 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813738 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813746 2563 flags.go:64] FLAG: --node-status-max-images="50" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813749 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813752 2563 flags.go:64] FLAG: --oom-score-adj="-999" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813755 2563 flags.go:64] FLAG: --pod-cidr="" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813758 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813763 2563 flags.go:64] FLAG: --pod-manifest-path="" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813767 2563 flags.go:64] FLAG: --pod-max-pids="-1" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813771 2563 flags.go:64] FLAG: --pods-per-core="0" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813774 2563 flags.go:64] FLAG: --port="10250" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813777 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813780 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eb28680c711466d8" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813783 2563 flags.go:64] FLAG: --qos-reserved="" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813787 2563 flags.go:64] FLAG: --read-only-port="10255" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813790 2563 flags.go:64] FLAG: --register-node="true" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813794 2563 flags.go:64] FLAG: --register-with-taints="" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813802 2563 flags.go:64] FLAG: --registry-burst="10" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813805 2563 flags.go:64] FLAG: --registry-qps="5" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813808 2563 flags.go:64] FLAG: --reserved-cpus="" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813811 2563 flags.go:64] FLAG: --reserved-memory="" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813815 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813818 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813822 2563 flags.go:64] FLAG: --rotate-certificates="false" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813825 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 17 19:06:30.821348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813828 2563 flags.go:64] FLAG: --runonce="false" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813831 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813834 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813837 2563 flags.go:64] FLAG: --seccomp-default="false" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813839 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813842 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813845 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813848 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813853 2563 flags.go:64] FLAG: --storage-driver-password="root" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813855 2563 flags.go:64] FLAG: --storage-driver-secure="false" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813859 2563 flags.go:64] FLAG: --storage-driver-table="stats" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813862 2563 flags.go:64] FLAG: --storage-driver-user="root" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813864 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813868 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813870 2563 flags.go:64] FLAG: --system-cgroups="" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813873 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813880 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813883 2563 flags.go:64] FLAG: --tls-cert-file="" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813885 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813890 2563 flags.go:64] FLAG: --tls-min-version="" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813892 2563 flags.go:64] FLAG: --tls-private-key-file="" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813897 2563 flags.go:64] FLAG: --topology-manager-policy="none" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813900 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813903 2563 flags.go:64] FLAG: --topology-manager-scope="container" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813906 2563 flags.go:64] FLAG: --v="2" Sep 17 19:06:30.822239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813910 2563 flags.go:64] FLAG: --version="false" Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813914 2563 flags.go:64] FLAG: --vmodule="" Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813918 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.813921 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814036 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814039 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814042 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814045 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814048 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814051 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814053 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814056 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814058 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814061 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814063 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814066 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814069 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814071 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814074 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814077 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:30.823130 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814079 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814082 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814085 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814089 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814091 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814094 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814097 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814099 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814104 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814107 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814109 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814113 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814125 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814128 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814131 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814133 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814136 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814139 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814141 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:30.823637 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814144 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814147 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814149 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814152 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814154 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814157 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814161 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814165 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814168 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814170 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814178 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814181 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814183 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814186 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814189 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814192 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814196 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814199 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814201 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814204 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:30.824163 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814207 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814211 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814213 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814216 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814218 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814221 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814224 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814226 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814229 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814232 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814235 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814237 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814240 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814242 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814245 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814248 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814250 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814253 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814256 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814258 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:30.824678 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814262 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814265 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814268 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814271 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814273 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814276 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814279 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814282 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814286 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814289 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814292 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814294 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814296 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814300 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814303 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814306 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814308 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814311 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814313 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814316 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:30.825270 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814318 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814321 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814323 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.814326 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.814332 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.824163 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.824180 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.824909 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.824917 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.824979 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.824984 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.824988 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.824991 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.824994 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.824997 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:30.825791 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825000 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825003 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825006 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825009 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825013 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825016 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825019 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825022 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825025 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825028 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825031 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825033 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825036 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825039 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825042 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825044 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825047 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825050 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825053 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825055 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:30.826157 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825058 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825060 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825070 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825074 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825077 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825079 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825082 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825085 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825087 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825090 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825092 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825095 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825098 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825100 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825103 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825106 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825108 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825111 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825115 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:30.826633 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825119 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825123 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825127 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825131 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825134 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825137 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825140 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825142 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825145 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825148 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825151 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825154 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825157 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825159 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825162 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825165 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825169 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825171 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825174 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:30.827116 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825178 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825182 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825186 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825189 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825192 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825194 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825198 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825201 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825204 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825206 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825209 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825212 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825214 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825217 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825219 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825223 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825226 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825228 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825231 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825234 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:30.827623 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825237 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825239 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825242 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825244 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825247 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825250 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825253 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825255 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825258 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825261 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825265 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825267 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825270 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825273 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825276 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:30.828159 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.825281 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825403 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825409 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825413 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825416 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825419 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825421 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825425 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825428 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825432 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825435 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825438 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825441 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825444 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825446 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825449 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825452 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825454 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825457 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:30.828522 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825460 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825463 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825465 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825468 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825471 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825474 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825476 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825479 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825482 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825484 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825487 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825489 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825492 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825496 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825499 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825502 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825505 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825508 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825510 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825513 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:30.829036 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825515 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825518 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825520 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825523 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825525 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825528 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825530 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825534 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825536 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825539 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825542 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825545 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825547 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825550 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825552 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825555 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825557 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825560 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825562 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825565 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:30.829551 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825568 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825570 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825573 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825575 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825578 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825580 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825583 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825585 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825588 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825590 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825593 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825596 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825598 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825601 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825603 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825606 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825608 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825611 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825614 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825616 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:30.830337 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825620 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825622 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825625 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825628 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825630 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825633 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825635 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825638 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825640 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825643 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825646 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825650 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825653 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825655 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825658 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825661 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825664 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825667 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825669 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:30.831027 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825672 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:30.831515 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:30.825675 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:30.831515 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.825680 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 19:06:30.831515 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.825841 2563 server.go:962] "Client rotation is on, will bootstrap in background" Sep 17 19:06:30.831515 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.828664 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 17 19:06:30.831515 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.829590 2563 server.go:1019] "Starting client certificate rotation" Sep 17 19:06:30.831515 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.829702 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 19:06:30.831515 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.829768 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 19:06:30.832170 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.832154 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 19:06:30.834800 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.834763 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 19:06:30.841520 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.841503 2563 log.go:25] "Validated CRI v1 runtime API" Sep 17 19:06:30.843198 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.843182 2563 log.go:25] "Validated CRI v1 image API" Sep 17 19:06:30.844080 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.844059 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 17 19:06:30.849425 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.849404 2563 fs.go:135] Filesystem UUIDs: map[6c89096d-9a9e-406a-9d6e-0bb9a3150757:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cab8c99a-0ec2-45ee-aeab-2f4c46cf5983:/dev/nvme0n1p3] Sep 17 19:06:30.849496 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.849422 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 17 19:06:30.857230 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.857113 2563 manager.go:216] Machine: {Timestamp:2026-09-17 19:06:30.855104557 +0000 UTC m=+0.318358732 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3112108 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e99caec06091f13b114911feb6bed SystemUUID:ec2e99ca-ec06-091f-13b1-14911feb6bed BootID:8c2c2549-cbce-4305-a4ed-b5598122d608 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c8:25:55:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c8:25:55:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:29:aa:9e:e2:08 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 17 19:06:30.857230 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.857221 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 17 19:06:30.857386 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.857314 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 17 19:06:30.860011 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.859992 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 19:06:30.860593 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.860570 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 19:06:30.860757 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.860599 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-16.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 17 19:06:30.862078 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.862066 2563 topology_manager.go:138] "Creating topology manager with none policy" Sep 17 19:06:30.862117 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.862081 2563 container_manager_linux.go:309] "Creating device plugin manager" Sep 17 19:06:30.862117 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.862089 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 19:06:30.862788 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.862778 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 19:06:30.864056 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.864047 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 17 19:06:30.867748 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.867736 2563 state_mem.go:36] "Initialized new in-memory state store" Sep 17 19:06:30.867870 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.867861 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 17 19:06:30.867906 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.867891 2563 kubelet.go:486] "Attempting to sync node with API server" Sep 17 19:06:30.867906 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.867901 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 17 19:06:30.867961 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.867916 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 17 19:06:30.867961 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.867925 2563 kubelet.go:398] "Adding apiserver pod source" Sep 17 19:06:30.867961 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.867935 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 17 19:06:30.868483 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.868341 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 19:06:30.868523 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.868510 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 19:06:30.870488 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.870472 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 17 19:06:30.873933 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.873918 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 17 19:06:30.873993 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.873957 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 17 19:06:30.876252 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876241 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 17 19:06:30.876290 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876257 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 17 19:06:30.876290 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876263 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 17 19:06:30.876290 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876269 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 17 19:06:30.876290 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876276 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 17 19:06:30.876290 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876282 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 17 19:06:30.876423 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876295 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 17 19:06:30.876423 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876302 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 17 19:06:30.876423 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876309 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 17 19:06:30.876423 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876315 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 17 19:06:30.876423 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876329 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 17 19:06:30.876423 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.876333 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 17 19:06:30.877122 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.877098 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 17 19:06:30.877122 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.877122 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 17 19:06:30.880028 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.880008 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-16.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 17 19:06:30.880194 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.880168 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 17 19:06:30.880194 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.880182 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 17 19:06:30.880956 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.880941 2563 server.go:1268] "Started kubelet" Sep 17 19:06:30.881116 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.881070 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 17 19:06:30.881157 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.881109 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 17 19:06:30.881196 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.881186 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 17 19:06:30.882258 ip-10-0-135-16 systemd[1]: Started Kubernetes Kubelet. Sep 17 19:06:30.882412 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.882395 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 17 19:06:30.884641 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.884624 2563 server.go:310] "Adding debug handlers to kubelet server" Sep 17 19:06:30.885671 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.885624 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 17 19:06:30.890060 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.889059 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-16.ec2.internal.18d6313190ec39d4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-16.ec2.internal,UID:ip-10-0-135-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-16.ec2.internal,},FirstTimestamp:2026-09-17 19:06:30.88090978 +0000 UTC m=+0.344163940,LastTimestamp:2026-09-17 19:06:30.88090978 +0000 UTC m=+0.344163940,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-16.ec2.internal,}" Sep 17 19:06:30.890984 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.890966 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 17 19:06:30.891502 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.891483 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 17 19:06:30.891603 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.891591 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found" Sep 17 19:06:30.892225 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892076 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 17 19:06:30.892225 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892105 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 17 19:06:30.892225 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892118 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 17 19:06:30.892537 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892275 2563 reconstruct.go:97] "Volume reconstruction finished" Sep 17 19:06:30.892537 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892283 2563 reconciler.go:29] "Reconciler: start to sync state" Sep 17 19:06:30.892537 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892399 2563 factory.go:55] Registering systemd factory Sep 17 19:06:30.892537 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892416 2563 factory.go:223] Registration of the systemd container factory successfully Sep 17 19:06:30.892807 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892672 2563 factory.go:153] Registering CRI-O factory Sep 17 19:06:30.892807 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892685 2563 factory.go:223] Registration of the crio container factory successfully Sep 17 19:06:30.892807 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892751 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 17 19:06:30.892807 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892767 2563 factory.go:103] Registering Raw factory Sep 17 19:06:30.892807 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.892780 2563 manager.go:1196] Started watching for new ooms in manager Sep 17 19:06:30.893153 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.893128 2563 manager.go:349] Starting recovery of all containers Sep 17 19:06:30.894707 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.894678 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 17 19:06:30.894821 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.894760 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 17 19:06:30.903975 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.903775 2563 manager.go:354] Recovery completed Sep 17 19:06:30.908409 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.908394 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:30.910344 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.910326 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:30.910409 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.910364 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:30.910409 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.910376 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:30.910903 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.910890 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 17 19:06:30.910903 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.910902 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 17 19:06:30.910988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.910917 2563 state_mem.go:36] "Initialized new in-memory state store" Sep 17 19:06:30.910988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.910895 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:30.911705 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.911688 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:30.911786 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.911734 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:30.911786 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.911748 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:30.912536 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.912464 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-16.ec2.internal.18d6313192ad631f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-16.ec2.internal,UID:ip-10-0-135-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-16.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-16.ec2.internal,},FirstTimestamp:2026-09-17 19:06:30.910346015 +0000 UTC m=+0.373600177,LastTimestamp:2026-09-17 19:06:30.910346015 +0000 UTC m=+0.373600177,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-16.ec2.internal,}" Sep 17 19:06:30.912897 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.912886 2563 policy_none.go:49] "None policy: Start" Sep 17 19:06:30.912937 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.912902 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 17 19:06:30.912937 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.912912 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 17 19:06:30.914691 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.914681 2563 policy_none.go:47] "Start" Sep 17 19:06:30.918397 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.918383 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j7zcn" Sep 17 19:06:30.925236 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.925161 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-16.ec2.internal.18d6313192adbffd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-16.ec2.internal,UID:ip-10-0-135-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-16.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-16.ec2.internal,},FirstTimestamp:2026-09-17 19:06:30.910369789 +0000 UTC m=+0.373623949,LastTimestamp:2026-09-17 19:06:30.910369789 +0000 UTC m=+0.373623949,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-16.ec2.internal,}" Sep 17 19:06:30.926667 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.926651 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j7zcn" Sep 17 19:06:30.955362 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.955343 2563 manager.go:337] "Starting Device Plugin manager" Sep 17 19:06:30.975791 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.955386 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 17 19:06:30.975791 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.955397 2563 server.go:87] "Starting device plugin registration server" Sep 17 19:06:30.975791 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.955781 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 17 19:06:30.975791 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.955793 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 17 19:06:30.975791 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.955905 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 17 19:06:30.975791 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.956000 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 17 19:06:30.975791 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:30.956009 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 17 19:06:30.975791 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.956521 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 17 19:06:30.975791 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:30.956578 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-16.ec2.internal\" not found" Sep 17 19:06:31.025659 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.025601 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 17 19:06:31.026956 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.026933 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 17 19:06:31.027062 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.026983 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 17 19:06:31.027062 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.027006 2563 kubelet.go:2448] "Starting kubelet main sync loop" Sep 17 19:06:31.027062 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.027042 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 17 19:06:31.029824 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.029801 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 19:06:31.055987 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.055906 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:31.056880 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.056865 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:31.056962 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.056895 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:31.056962 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.056905 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:31.056962 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.056928 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.067005 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.066986 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.067055 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.067010 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-16.ec2.internal\": node \"ip-10-0-135-16.ec2.internal\" not found" Sep 17 19:06:31.116086 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.116061 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found" Sep 17 19:06:31.128072 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.128052 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal"] Sep 17 19:06:31.128135 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.128115 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:31.129574 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.129557 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:31.129655 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.129588 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:31.129655 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.129598 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:31.131774 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.131762 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:31.131877 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.131864 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.131925 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.131891 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:31.133446 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.133430 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:31.133522 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.133462 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:31.133522 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.133474 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:31.133522 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.133432 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:31.133636 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.133530 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:31.133636 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.133542 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:31.135634 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.135620 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.135701 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.135647 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:31.136430 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.136415 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:31.136499 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.136444 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:31.136499 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.136453 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:31.159384 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.159360 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-16.ec2.internal\" not found" node="ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.163691 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.163676 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-16.ec2.internal\" not found" node="ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.193848 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.193823 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4563741808a577ad9736e2cd839b878e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"4563741808a577ad9736e2cd839b878e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.193931 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.193855 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4563741808a577ad9736e2cd839b878e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"4563741808a577ad9736e2cd839b878e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.193931 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.193894 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aac0a2a35937da63c2df9c101e366883-config\") pod \"kube-apiserver-proxy-ip-10-0-135-16.ec2.internal\" (UID: \"aac0a2a35937da63c2df9c101e366883\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.216223 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.216203 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found" Sep 17 19:06:31.294763 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.294732 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4563741808a577ad9736e2cd839b878e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"4563741808a577ad9736e2cd839b878e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.294851 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.294766 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4563741808a577ad9736e2cd839b878e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"4563741808a577ad9736e2cd839b878e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.294851 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.294786 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aac0a2a35937da63c2df9c101e366883-config\") pod \"kube-apiserver-proxy-ip-10-0-135-16.ec2.internal\" (UID: \"aac0a2a35937da63c2df9c101e366883\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.294851 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.294848 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aac0a2a35937da63c2df9c101e366883-config\") pod \"kube-apiserver-proxy-ip-10-0-135-16.ec2.internal\" (UID: \"aac0a2a35937da63c2df9c101e366883\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.294956 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.294854 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4563741808a577ad9736e2cd839b878e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"4563741808a577ad9736e2cd839b878e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.294956 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.294897 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4563741808a577ad9736e2cd839b878e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"4563741808a577ad9736e2cd839b878e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.316905 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.316832 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found" Sep 17 19:06:31.417635 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.417594 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found" Sep 17 19:06:31.462750 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.462694 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.468170 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.467999 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.518479 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.518436 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found" Sep 17 19:06:31.618991 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.618899 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found" Sep 17 19:06:31.719432 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.719396 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found" Sep 17 19:06:31.819940 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:31.819907 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found" Sep 17 19:06:31.830089 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.830065 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 17 19:06:31.830245 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.830211 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 19:06:31.897849 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.897776 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 19:06:31.901268 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.901248 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 19:06:31.932235 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.932202 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 19:01:32 +0000 UTC" deadline="2028-02-20 08:39:19.435007831 +0000 UTC" Sep 17 19:06:31.932235 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.932231 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12493h32m47.502780175s" Sep 17 19:06:31.956123 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.956099 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 19:06:31.992478 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:31.992444 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" Sep 17 19:06:31.994082 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:31.994054 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4563741808a577ad9736e2cd839b878e.slice/crio-2cd681bfe3b3dc34c0ef9a7394949ff3258d89342d87e41a7ffa9d7e11d41536 WatchSource:0}: Error finding container 2cd681bfe3b3dc34c0ef9a7394949ff3258d89342d87e41a7ffa9d7e11d41536: Status 404 returned error can't find the container with id 2cd681bfe3b3dc34c0ef9a7394949ff3258d89342d87e41a7ffa9d7e11d41536 Sep 17 19:06:31.994419 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:31.994403 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaac0a2a35937da63c2df9c101e366883.slice/crio-5e3f3a7831c56448af8056021cbe4dc02b936e8591fb1c8e2915171edf38b55b WatchSource:0}: Error finding container 5e3f3a7831c56448af8056021cbe4dc02b936e8591fb1c8e2915171edf38b55b: Status 404 returned error can't find the container with id 5e3f3a7831c56448af8056021cbe4dc02b936e8591fb1c8e2915171edf38b55b Sep 17 19:06:32.000609 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:32.000588 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 19:06:32.024204 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:32.024182 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 19:06:32.025207 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:32.025193 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" Sep 17 19:06:32.030476 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:32.030436 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" event={"ID":"4563741808a577ad9736e2cd839b878e","Type":"ContainerStarted","Data":"2cd681bfe3b3dc34c0ef9a7394949ff3258d89342d87e41a7ffa9d7e11d41536"} Sep 17 19:06:32.031551 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:32.031530 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" event={"ID":"aac0a2a35937da63c2df9c101e366883","Type":"ContainerStarted","Data":"5e3f3a7831c56448af8056021cbe4dc02b936e8591fb1c8e2915171edf38b55b"} Sep 17 19:06:32.041570 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:32.041557 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 19:06:33.904619 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:33.904585 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 19:06:34.070668 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.070631 2563 apiserver.go:52] "Watching apiserver" Sep 17 19:06:34.077735 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.077705 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 17 19:06:34.078737 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.078703 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-r5nkm","openshift-multus/network-metrics-daemon-6mspz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8","openshift-dns/node-resolver-l6h2d","openshift-image-registry/node-ca-q89g2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal","openshift-multus/multus-additional-cni-plugins-f9wcr","openshift-network-diagnostics/network-check-target-z7q6d","openshift-network-operator/iptables-alerter-69wbb","openshift-ovn-kubernetes/ovnkube-node-m5854","kube-system/global-pull-secret-syncer-xvjpg","kube-system/konnectivity-agent-94j6j","kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal","openshift-cluster-node-tuning-operator/tuned-dc7vl"] Sep 17 19:06:34.081456 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.081431 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.083525 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.083497 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:34.083623 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.083587 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:34.084742 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.084482 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 17 19:06:34.084742 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.084560 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 17 19:06:34.084947 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.084913 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 17 19:06:34.085023 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.085006 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-644x6\"" Sep 17 19:06:34.085260 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.085246 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 17 19:06:34.085918 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.085897 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.087124 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.086051 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 17 19:06:34.088436 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.088112 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l6h2d" Sep 17 19:06:34.088872 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.088844 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 17 19:06:34.088992 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.088973 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 17 19:06:34.089202 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.089187 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 17 19:06:34.089312 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.089292 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87hmb\"" Sep 17 19:06:34.090167 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.090149 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 17 19:06:34.090417 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.090399 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q89g2" Sep 17 19:06:34.090774 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.090424 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7ldvk\"" Sep 17 19:06:34.090880 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.090706 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 17 19:06:34.092718 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.092701 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.092844 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.092826 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 17 19:06:34.093173 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.093152 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 17 19:06:34.093302 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.093286 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 17 19:06:34.093386 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.093342 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2mj9k\"" Sep 17 19:06:34.093443 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.093289 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 17 19:06:34.094912 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.094897 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-69wbb" Sep 17 19:06:34.096140 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.096121 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 17 19:06:34.096437 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.096422 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 17 19:06:34.097662 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.097643 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:34.097733 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.097708 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:34.099502 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.099483 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 17 19:06:34.099590 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.099499 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pb2tw\"" Sep 17 19:06:34.099590 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.099540 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r469w\"" Sep 17 19:06:34.099927 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.099909 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:34.100081 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.100056 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:34.100545 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.100525 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 17 19:06:34.100611 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.100562 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 17 19:06:34.103238 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103221 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.103342 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103234 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f2lk\" (UniqueName: \"kubernetes.io/projected/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-kube-api-access-5f2lk\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.103342 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103288 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-kubernetes\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.103342 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103312 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-sysctl-conf\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.103342 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103337 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf2fr\" (UniqueName: \"kubernetes.io/projected/5827bc3d-1d3b-4b60-9217-ffe0998fc733-kube-api-access-nf2fr\") pod \"iptables-alerter-69wbb\" (UID: \"5827bc3d-1d3b-4b60-9217-ffe0998fc733\") " pod="openshift-network-operator/iptables-alerter-69wbb" Sep 17 19:06:34.103550 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103365 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b46af12-297a-4da0-b3f3-730424604854-cnibin\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.103550 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103395 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b46af12-297a-4da0-b3f3-730424604854-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.103550 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103429 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:34.103550 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103467 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-device-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.103550 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103498 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac28db00-3290-4731-ab0c-9963ef1d6560-hosts-file\") pod \"node-resolver-l6h2d\" (UID: \"ac28db00-3290-4731-ab0c-9963ef1d6560\") " pod="openshift-dns/node-resolver-l6h2d" Sep 17 19:06:34.103550 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103530 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxbtk\" (UniqueName: \"kubernetes.io/projected/8b46af12-297a-4da0-b3f3-730424604854-kube-api-access-gxbtk\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.105055 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.103554 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-socket-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.105158 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105084 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-registration-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.105158 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105144 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5827bc3d-1d3b-4b60-9217-ffe0998fc733-host-slash\") pod \"iptables-alerter-69wbb\" (UID: \"5827bc3d-1d3b-4b60-9217-ffe0998fc733\") " pod="openshift-network-operator/iptables-alerter-69wbb" Sep 17 19:06:34.105271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105176 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-dbus\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:34.105271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105241 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/76086d58-ef7d-4a2c-9614-c15931bac3df-serviceca\") pod \"node-ca-q89g2\" (UID: \"76086d58-ef7d-4a2c-9614-c15931bac3df\") " pod="openshift-image-registry/node-ca-q89g2" Sep 17 19:06:34.105417 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105278 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t48h2\" (UniqueName: \"kubernetes.io/projected/76086d58-ef7d-4a2c-9614-c15931bac3df-kube-api-access-t48h2\") pod \"node-ca-q89g2\" (UID: \"76086d58-ef7d-4a2c-9614-c15931bac3df\") " pod="openshift-image-registry/node-ca-q89g2" Sep 17 19:06:34.105472 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105423 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-lib-modules\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.105472 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105443 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-var-lib-kubelet\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.105472 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105464 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b46af12-297a-4da0-b3f3-730424604854-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.105620 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76086d58-ef7d-4a2c-9614-c15931bac3df-host\") pod \"node-ca-q89g2\" (UID: \"76086d58-ef7d-4a2c-9614-c15931bac3df\") " pod="openshift-image-registry/node-ca-q89g2" Sep 17 19:06:34.105620 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105540 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-systemd\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.105620 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105555 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wxj2\" (UniqueName: \"kubernetes.io/projected/532a9820-a0c8-4dec-a697-9e7d15057df6-kube-api-access-4wxj2\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.105620 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105590 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5827bc3d-1d3b-4b60-9217-ffe0998fc733-iptables-alerter-script\") pod \"iptables-alerter-69wbb\" (UID: \"5827bc3d-1d3b-4b60-9217-ffe0998fc733\") " pod="openshift-network-operator/iptables-alerter-69wbb" Sep 17 19:06:34.105819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105622 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b46af12-297a-4da0-b3f3-730424604854-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.105819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105640 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-sysconfig\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.105819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105655 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac28db00-3290-4731-ab0c-9963ef1d6560-tmp-dir\") pod \"node-resolver-l6h2d\" (UID: \"ac28db00-3290-4731-ab0c-9963ef1d6560\") " pod="openshift-dns/node-resolver-l6h2d" Sep 17 19:06:34.105819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105747 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b46af12-297a-4da0-b3f3-730424604854-system-cni-dir\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.105819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105766 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-kubelet-config\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:34.105819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105791 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-etc-selinux\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.105819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105816 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-modprobe-d\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.106184 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-sysctl-d\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.106184 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105889 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-sys\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.106184 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105907 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-host\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.106184 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105940 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/532a9820-a0c8-4dec-a697-9e7d15057df6-tmp\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.106184 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.105977 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b46af12-297a-4da0-b3f3-730424604854-cni-binary-copy\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.106184 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.106008 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-kubelet-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.106184 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.106022 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-sys-fs\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.106184 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.106038 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-run\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.106184 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.106063 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-tuned\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.106184 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.106081 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8zjv\" (UniqueName: \"kubernetes.io/projected/ac28db00-3290-4731-ab0c-9963ef1d6560-kube-api-access-g8zjv\") pod \"node-resolver-l6h2d\" (UID: \"ac28db00-3290-4731-ab0c-9963ef1d6560\") " pod="openshift-dns/node-resolver-l6h2d" Sep 17 19:06:34.107757 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.107739 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.108940 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.108921 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 17 19:06:34.109035 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.108968 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 17 19:06:34.109474 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.109448 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 17 19:06:34.109576 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.109552 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9xlxm\"" Sep 17 19:06:34.109847 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.109829 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 17 19:06:34.109847 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.109841 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 17 19:06:34.109967 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.109941 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:06:34.110028 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.109829 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 17 19:06:34.111250 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.111230 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xg272\"" Sep 17 19:06:34.113801 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.113778 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-snxf2\"" Sep 17 19:06:34.113801 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.113785 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 17 19:06:34.114062 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.114042 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 17 19:06:34.114741 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.114721 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 17 19:06:34.120035 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.119858 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 19:06:34.162666 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.162641 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2fcd9" Sep 17 19:06:34.173211 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.173186 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2fcd9" Sep 17 19:06:34.194842 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.194822 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 17 19:06:34.207263 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207235 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-sysconfig\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.207406 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207280 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-run-openvswitch\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.207406 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207308 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-cni-netd\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.207406 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207333 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-kubelet-config\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:34.207406 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207372 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-sysconfig\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.207406 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207376 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/532a9820-a0c8-4dec-a697-9e7d15057df6-tmp\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.207652 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207432 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qfg7\" (UniqueName: \"kubernetes.io/projected/5ce6506b-9752-4fbe-9622-29ecf358b465-kube-api-access-4qfg7\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.207652 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207466 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b46af12-297a-4da0-b3f3-730424604854-cni-binary-copy\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.207652 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207493 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-kubelet-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.207652 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207534 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-run\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.207652 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207557 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-var-lib-cni-bin\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.207652 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207582 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-kubelet\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.207652 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207605 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-systemd-units\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.207652 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207649 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f2lk\" (UniqueName: \"kubernetes.io/projected/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-kube-api-access-5f2lk\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.208004 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207673 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-kubernetes\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.208004 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207698 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf2fr\" (UniqueName: \"kubernetes.io/projected/5827bc3d-1d3b-4b60-9217-ffe0998fc733-kube-api-access-nf2fr\") pod \"iptables-alerter-69wbb\" (UID: \"5827bc3d-1d3b-4b60-9217-ffe0998fc733\") " pod="openshift-network-operator/iptables-alerter-69wbb" Sep 17 19:06:34.208004 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207722 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-system-cni-dir\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.208004 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207747 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-run-netns\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.208004 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207773 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:34.208004 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207801 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-device-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.208004 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207805 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-kubelet-config\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:34.208004 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207827 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac28db00-3290-4731-ab0c-9963ef1d6560-hosts-file\") pod \"node-resolver-l6h2d\" (UID: \"ac28db00-3290-4731-ab0c-9963ef1d6560\") " pod="openshift-dns/node-resolver-l6h2d" Sep 17 19:06:34.208004 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207857 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxbtk\" (UniqueName: \"kubernetes.io/projected/8b46af12-297a-4da0-b3f3-730424604854-kube-api-access-gxbtk\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.208004 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207970 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac28db00-3290-4731-ab0c-9963ef1d6560-hosts-file\") pod \"node-resolver-l6h2d\" (UID: \"ac28db00-3290-4731-ab0c-9963ef1d6560\") " pod="openshift-dns/node-resolver-l6h2d" Sep 17 19:06:34.208004 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.207983 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-run\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208135 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-kubelet-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208121 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208214 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-socket-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208253 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-kubernetes\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208266 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5827bc3d-1d3b-4b60-9217-ffe0998fc733-host-slash\") pod \"iptables-alerter-69wbb\" (UID: \"5827bc3d-1d3b-4b60-9217-ffe0998fc733\") " pod="openshift-network-operator/iptables-alerter-69wbb" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208300 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vflj9\" (UniqueName: \"kubernetes.io/projected/431387c9-5a3c-44d5-bad2-94a08bc73700-kube-api-access-vflj9\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208325 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-cni-dir\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208361 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-socket-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-slash\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208395 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-log-socket\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208408 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5827bc3d-1d3b-4b60-9217-ffe0998fc733-host-slash\") pod \"iptables-alerter-69wbb\" (UID: \"5827bc3d-1d3b-4b60-9217-ffe0998fc733\") " pod="openshift-network-operator/iptables-alerter-69wbb" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208420 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/76086d58-ef7d-4a2c-9614-c15931bac3df-serviceca\") pod \"node-ca-q89g2\" (UID: \"76086d58-ef7d-4a2c-9614-c15931bac3df\") " pod="openshift-image-registry/node-ca-q89g2" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.208458 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208467 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t48h2\" (UniqueName: \"kubernetes.io/projected/76086d58-ef7d-4a2c-9614-c15931bac3df-kube-api-access-t48h2\") pod \"node-ca-q89g2\" (UID: \"76086d58-ef7d-4a2c-9614-c15931bac3df\") " pod="openshift-image-registry/node-ca-q89g2" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208496 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-var-lib-kubelet\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.208554 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret podName:7d66c218-c6ed-4ca6-a49a-47bdc94fef2d nodeName:}" failed. No retries permitted until 2026-09-17 19:06:34.708518639 +0000 UTC m=+2.970392027 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret") pod "global-pull-secret-syncer-xvjpg" (UID: "7d66c218-c6ed-4ca6-a49a-47bdc94fef2d") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:34.208598 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208567 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-var-lib-kubelet\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208573 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hx9f\" (UniqueName: \"kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f\") pod \"network-check-target-z7q6d\" (UID: \"357e149f-654a-4188-9229-1a440c1c30a9\") " pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208602 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-var-lib-cni-multus\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208672 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-run-multus-certs\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208726 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b46af12-297a-4da0-b3f3-730424604854-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208757 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b46af12-297a-4da0-b3f3-730424604854-cni-binary-copy\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208775 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-systemd\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208805 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wxj2\" (UniqueName: \"kubernetes.io/projected/532a9820-a0c8-4dec-a697-9e7d15057df6-kube-api-access-4wxj2\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208904 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b46af12-297a-4da0-b3f3-730424604854-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208970 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/76086d58-ef7d-4a2c-9614-c15931bac3df-serviceca\") pod \"node-ca-q89g2\" (UID: \"76086d58-ef7d-4a2c-9614-c15931bac3df\") " pod="openshift-image-registry/node-ca-q89g2" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208974 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-systemd\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.208974 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-device-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209002 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5827bc3d-1d3b-4b60-9217-ffe0998fc733-iptables-alerter-script\") pod \"iptables-alerter-69wbb\" (UID: \"5827bc3d-1d3b-4b60-9217-ffe0998fc733\") " pod="openshift-network-operator/iptables-alerter-69wbb" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209038 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-cnibin\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209074 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-daemon-config\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209128 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-run-systemd\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209170 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b46af12-297a-4da0-b3f3-730424604854-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.209320 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209202 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac28db00-3290-4731-ab0c-9963ef1d6560-tmp-dir\") pod \"node-resolver-l6h2d\" (UID: \"ac28db00-3290-4731-ab0c-9963ef1d6560\") " pod="openshift-dns/node-resolver-l6h2d" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209313 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c13b5773-a330-4e28-8c36-dbed2847dd86-ovn-node-metrics-cert\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209393 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b46af12-297a-4da0-b3f3-730424604854-system-cni-dir\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209432 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac28db00-3290-4731-ab0c-9963ef1d6560-tmp-dir\") pod \"node-resolver-l6h2d\" (UID: \"ac28db00-3290-4731-ab0c-9963ef1d6560\") " pod="openshift-dns/node-resolver-l6h2d" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209447 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b46af12-297a-4da0-b3f3-730424604854-system-cni-dir\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209520 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-etc-selinux\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209562 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-modprobe-d\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209568 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5827bc3d-1d3b-4b60-9217-ffe0998fc733-iptables-alerter-script\") pod \"iptables-alerter-69wbb\" (UID: \"5827bc3d-1d3b-4b60-9217-ffe0998fc733\") " pod="openshift-network-operator/iptables-alerter-69wbb" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209614 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-etc-selinux\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209592 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-sysctl-d\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209663 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-sys\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209679 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-sysctl-d\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209687 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-host\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209712 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-socket-dir-parent\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209717 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-modprobe-d\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209740 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-sys\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209744 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-conf-dir\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.210040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209768 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-sys-fs\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209772 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-host\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209783 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-tuned\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209799 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8zjv\" (UniqueName: \"kubernetes.io/projected/ac28db00-3290-4731-ab0c-9963ef1d6560-kube-api-access-g8zjv\") pod \"node-resolver-l6h2d\" (UID: \"ac28db00-3290-4731-ab0c-9963ef1d6560\") " pod="openshift-dns/node-resolver-l6h2d" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209833 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-sys-fs\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209836 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-run-k8s-cni-cncf-io\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209859 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-etc-kubernetes\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209880 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-run-ovn\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209894 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-run-ovn-kubernetes\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209909 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-cni-bin\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.209925 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-sysctl-conf\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210065 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-var-lib-openvswitch\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b46af12-297a-4da0-b3f3-730424604854-cnibin\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210130 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b46af12-297a-4da0-b3f3-730424604854-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210301 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b46af12-297a-4da0-b3f3-730424604854-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210316 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ce6506b-9752-4fbe-9622-29ecf358b465-cni-binary-copy\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210232 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b46af12-297a-4da0-b3f3-730424604854-cnibin\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.211034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210191 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-sysctl-conf\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210359 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58883cfb-501e-4277-9ece-f19ed6452fb7-agent-certs\") pod \"konnectivity-agent-94j6j\" (UID: \"58883cfb-501e-4277-9ece-f19ed6452fb7\") " pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210388 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-registration-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210412 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-run-netns\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210449 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-node-log\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210466 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-registration-dir\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210470 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c13b5773-a330-4e28-8c36-dbed2847dd86-ovnkube-config\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210528 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c13b5773-a330-4e28-8c36-dbed2847dd86-env-overrides\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210556 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c13b5773-a330-4e28-8c36-dbed2847dd86-ovnkube-script-lib\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210579 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58883cfb-501e-4277-9ece-f19ed6452fb7-konnectivity-ca\") pod \"konnectivity-agent-94j6j\" (UID: \"58883cfb-501e-4277-9ece-f19ed6452fb7\") " pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210618 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-dbus\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210652 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b46af12-297a-4da0-b3f3-730424604854-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210660 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-lib-modules\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210691 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210713 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-var-lib-kubelet\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210734 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-hostroot\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210756 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-etc-openvswitch\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.211884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210765 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-dbus\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:34.212628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210781 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76086d58-ef7d-4a2c-9614-c15931bac3df-host\") pod \"node-ca-q89g2\" (UID: \"76086d58-ef7d-4a2c-9614-c15931bac3df\") " pod="openshift-image-registry/node-ca-q89g2" Sep 17 19:06:34.212628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210804 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.212628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210821 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/532a9820-a0c8-4dec-a697-9e7d15057df6-lib-modules\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.212628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210849 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5kmv\" (UniqueName: \"kubernetes.io/projected/c13b5773-a330-4e28-8c36-dbed2847dd86-kube-api-access-r5kmv\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.212628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.210841 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76086d58-ef7d-4a2c-9614-c15931bac3df-host\") pod \"node-ca-q89g2\" (UID: \"76086d58-ef7d-4a2c-9614-c15931bac3df\") " pod="openshift-image-registry/node-ca-q89g2" Sep 17 19:06:34.212628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.212363 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/532a9820-a0c8-4dec-a697-9e7d15057df6-tmp\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.212628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.212558 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/532a9820-a0c8-4dec-a697-9e7d15057df6-etc-tuned\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.220763 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.220741 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t48h2\" (UniqueName: \"kubernetes.io/projected/76086d58-ef7d-4a2c-9614-c15931bac3df-kube-api-access-t48h2\") pod \"node-ca-q89g2\" (UID: \"76086d58-ef7d-4a2c-9614-c15931bac3df\") " pod="openshift-image-registry/node-ca-q89g2" Sep 17 19:06:34.220904 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.220887 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f2lk\" (UniqueName: \"kubernetes.io/projected/6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247-kube-api-access-5f2lk\") pod \"aws-ebs-csi-driver-node-prjm8\" (UID: \"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.221181 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.221159 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxbtk\" (UniqueName: \"kubernetes.io/projected/8b46af12-297a-4da0-b3f3-730424604854-kube-api-access-gxbtk\") pod \"multus-additional-cni-plugins-f9wcr\" (UID: \"8b46af12-297a-4da0-b3f3-730424604854\") " pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.222262 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.222219 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf2fr\" (UniqueName: \"kubernetes.io/projected/5827bc3d-1d3b-4b60-9217-ffe0998fc733-kube-api-access-nf2fr\") pod \"iptables-alerter-69wbb\" (UID: \"5827bc3d-1d3b-4b60-9217-ffe0998fc733\") " pod="openshift-network-operator/iptables-alerter-69wbb" Sep 17 19:06:34.222375 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.222348 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wxj2\" (UniqueName: \"kubernetes.io/projected/532a9820-a0c8-4dec-a697-9e7d15057df6-kube-api-access-4wxj2\") pod \"tuned-dc7vl\" (UID: \"532a9820-a0c8-4dec-a697-9e7d15057df6\") " pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.225039 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.225018 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8zjv\" (UniqueName: \"kubernetes.io/projected/ac28db00-3290-4731-ab0c-9963ef1d6560-kube-api-access-g8zjv\") pod \"node-resolver-l6h2d\" (UID: \"ac28db00-3290-4731-ab0c-9963ef1d6560\") " pod="openshift-dns/node-resolver-l6h2d" Sep 17 19:06:34.311817 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.311782 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vflj9\" (UniqueName: \"kubernetes.io/projected/431387c9-5a3c-44d5-bad2-94a08bc73700-kube-api-access-vflj9\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:34.312016 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.311869 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-cni-dir\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.312016 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.311929 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-slash\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.312016 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.311963 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-log-socket\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.312321 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.312298 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-cni-dir\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.312432 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.312396 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-slash\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.312486 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.312459 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-log-socket\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.312582 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.312563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hx9f\" (UniqueName: \"kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f\") pod \"network-check-target-z7q6d\" (UID: \"357e149f-654a-4188-9229-1a440c1c30a9\") " pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:34.312720 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.312702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-var-lib-cni-multus\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.312775 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.312767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-run-multus-certs\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.312825 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.312799 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-cnibin\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.312878 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.312831 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-daemon-config\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.312878 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.312861 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-run-systemd\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.312961 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.312917 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c13b5773-a330-4e28-8c36-dbed2847dd86-ovn-node-metrics-cert\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.313008 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.312988 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-socket-dir-parent\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.313151 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-conf-dir\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.313271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313220 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-run-systemd\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.313342 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313263 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-var-lib-cni-multus\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.313342 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313289 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-run-k8s-cni-cncf-io\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.313342 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313327 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-etc-kubernetes\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.313490 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313340 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-socket-dir-parent\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.313490 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313343 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-run-multus-certs\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.313490 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313358 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-run-ovn\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.313490 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313407 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-cnibin\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.313490 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313397 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-conf-dir\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.313725 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313513 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-etc-kubernetes\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.313725 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313554 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-run-ovn-kubernetes\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.313725 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313589 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-cni-bin\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.313725 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313622 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-var-lib-openvswitch\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.313725 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313648 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ce6506b-9752-4fbe-9622-29ecf358b465-cni-binary-copy\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.313725 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313675 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58883cfb-501e-4277-9ece-f19ed6452fb7-agent-certs\") pod \"konnectivity-agent-94j6j\" (UID: \"58883cfb-501e-4277-9ece-f19ed6452fb7\") " pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:06:34.313725 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313705 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-run-netns\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313734 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-node-log\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313762 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c13b5773-a330-4e28-8c36-dbed2847dd86-ovnkube-config\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313784 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c13b5773-a330-4e28-8c36-dbed2847dd86-env-overrides\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313812 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c13b5773-a330-4e28-8c36-dbed2847dd86-ovnkube-script-lib\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313841 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58883cfb-501e-4277-9ece-f19ed6452fb7-konnectivity-ca\") pod \"konnectivity-agent-94j6j\" (UID: \"58883cfb-501e-4277-9ece-f19ed6452fb7\") " pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313871 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313880 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-run-ovn-kubernetes\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313898 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-var-lib-kubelet\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313925 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-hostroot\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313956 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-etc-openvswitch\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313943 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-cni-bin\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.313591 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-run-ovn\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314004 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314013 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5ce6506b-9752-4fbe-9622-29ecf358b465-multus-daemon-config\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314037 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5kmv\" (UniqueName: \"kubernetes.io/projected/c13b5773-a330-4e28-8c36-dbed2847dd86-kube-api-access-r5kmv\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314054 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314030 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-var-lib-openvswitch\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314116 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-run-openvswitch\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314156 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-run-netns\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314181 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-node-log\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314178 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-run-openvswitch\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314250 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-var-lib-kubelet\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314268 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314315 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-cni-netd\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314352 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qfg7\" (UniqueName: \"kubernetes.io/projected/5ce6506b-9752-4fbe-9622-29ecf358b465-kube-api-access-4qfg7\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314383 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-var-lib-cni-bin\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314406 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-kubelet\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314433 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-systemd-units\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314503 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-system-cni-dir\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314559 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-run-netns\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314714 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-etc-openvswitch\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314751 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-hostroot\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314762 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-run-netns\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.314869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314797 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ce6506b-9752-4fbe-9622-29ecf358b465-cni-binary-copy\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.315692 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.314990 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-cni-netd\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.315692 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.315069 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-var-lib-cni-bin\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.315692 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.315162 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-host-kubelet\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.315692 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.315210 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c13b5773-a330-4e28-8c36-dbed2847dd86-systemd-units\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.315692 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.315241 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c13b5773-a330-4e28-8c36-dbed2847dd86-env-overrides\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.315692 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.315297 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-system-cni-dir\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.315692 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.315613 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58883cfb-501e-4277-9ece-f19ed6452fb7-konnectivity-ca\") pod \"konnectivity-agent-94j6j\" (UID: \"58883cfb-501e-4277-9ece-f19ed6452fb7\") " pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:06:34.316000 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.315701 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5ce6506b-9752-4fbe-9622-29ecf358b465-host-run-k8s-cni-cncf-io\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.316000 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.315796 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c13b5773-a330-4e28-8c36-dbed2847dd86-ovnkube-config\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.316000 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.315861 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:34.316521 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.316171 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c13b5773-a330-4e28-8c36-dbed2847dd86-ovnkube-script-lib\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.316521 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.316194 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs podName:431387c9-5a3c-44d5-bad2-94a08bc73700 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:34.815938417 +0000 UTC m=+3.077811816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs") pod "network-metrics-daemon-6mspz" (UID: "431387c9-5a3c-44d5-bad2-94a08bc73700") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:34.316878 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.316855 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c13b5773-a330-4e28-8c36-dbed2847dd86-ovn-node-metrics-cert\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.318828 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.318805 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58883cfb-501e-4277-9ece-f19ed6452fb7-agent-certs\") pod \"konnectivity-agent-94j6j\" (UID: \"58883cfb-501e-4277-9ece-f19ed6452fb7\") " pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:06:34.326425 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.326399 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:34.326505 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.326429 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:34.326505 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.326443 2563 projected.go:196] Error preparing data for projected volume kube-api-access-2hx9f for pod openshift-network-diagnostics/network-check-target-z7q6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:34.326505 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.326499 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f podName:357e149f-654a-4188-9229-1a440c1c30a9 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:34.826482566 +0000 UTC m=+3.088355973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2hx9f" (UniqueName: "kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f") pod "network-check-target-z7q6d" (UID: "357e149f-654a-4188-9229-1a440c1c30a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:34.327306 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.327273 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vflj9\" (UniqueName: \"kubernetes.io/projected/431387c9-5a3c-44d5-bad2-94a08bc73700-kube-api-access-vflj9\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:34.327487 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.327463 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5kmv\" (UniqueName: \"kubernetes.io/projected/c13b5773-a330-4e28-8c36-dbed2847dd86-kube-api-access-r5kmv\") pod \"ovnkube-node-m5854\" (UID: \"c13b5773-a330-4e28-8c36-dbed2847dd86\") " pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.328786 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.328767 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qfg7\" (UniqueName: \"kubernetes.io/projected/5ce6506b-9752-4fbe-9622-29ecf358b465-kube-api-access-4qfg7\") pod \"multus-r5nkm\" (UID: \"5ce6506b-9752-4fbe-9622-29ecf358b465\") " pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.395841 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.395809 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" Sep 17 19:06:34.402324 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.402302 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" Sep 17 19:06:34.412166 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.412088 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l6h2d" Sep 17 19:06:34.419857 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.419826 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q89g2" Sep 17 19:06:34.428808 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.428784 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" Sep 17 19:06:34.436424 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.436398 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-69wbb" Sep 17 19:06:34.445918 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.445895 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:34.453655 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.453637 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r5nkm" Sep 17 19:06:34.459180 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.459163 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:06:34.716697 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.716612 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:34.716871 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.716771 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:34.716871 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.716839 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret podName:7d66c218-c6ed-4ca6-a49a-47bdc94fef2d nodeName:}" failed. No retries permitted until 2026-09-17 19:06:35.716820414 +0000 UTC m=+3.978693801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret") pod "global-pull-secret-syncer-xvjpg" (UID: "7d66c218-c6ed-4ca6-a49a-47bdc94fef2d") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:34.817412 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.817380 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:34.817569 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.817531 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:34.817609 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.817597 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs podName:431387c9-5a3c-44d5-bad2-94a08bc73700 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:35.817582894 +0000 UTC m=+4.079456292 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs") pod "network-metrics-daemon-6mspz" (UID: "431387c9-5a3c-44d5-bad2-94a08bc73700") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:34.918524 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:34.918489 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hx9f\" (UniqueName: \"kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f\") pod \"network-check-target-z7q6d\" (UID: \"357e149f-654a-4188-9229-1a440c1c30a9\") " pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:34.918700 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.918678 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:34.918757 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.918707 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:34.918757 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.918718 2563 projected.go:196] Error preparing data for projected volume kube-api-access-2hx9f for pod openshift-network-diagnostics/network-check-target-z7q6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:34.918832 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:34.918779 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f podName:357e149f-654a-4188-9229-1a440c1c30a9 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:35.918758757 +0000 UTC m=+4.180632143 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2hx9f" (UniqueName: "kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f") pod "network-check-target-z7q6d" (UID: "357e149f-654a-4188-9229-1a440c1c30a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:35.152532 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:35.152492 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod532a9820_a0c8_4dec_a697_9e7d15057df6.slice/crio-cc6f4e3c486a47c1877a9724343fb847e3832e13b0dee9bb8bde340c9bbadcd6 WatchSource:0}: Error finding container cc6f4e3c486a47c1877a9724343fb847e3832e13b0dee9bb8bde340c9bbadcd6: Status 404 returned error can't find the container with id cc6f4e3c486a47c1877a9724343fb847e3832e13b0dee9bb8bde340c9bbadcd6 Sep 17 19:06:35.157148 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:35.157089 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7ec5b3_e475_40c6_b4eb_97a3fe2ef247.slice/crio-1c5e34b0a869ebd24339f24714639e7fd2726136aec81903cc8dbacc66f83894 WatchSource:0}: Error finding container 1c5e34b0a869ebd24339f24714639e7fd2726136aec81903cc8dbacc66f83894: Status 404 returned error can't find the container with id 1c5e34b0a869ebd24339f24714639e7fd2726136aec81903cc8dbacc66f83894 Sep 17 19:06:35.157769 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:35.157744 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac28db00_3290_4731_ab0c_9963ef1d6560.slice/crio-31bbad8d6eea61b1ac0a43d3b313aed602f2c0b9219aa91065d1573941386f0c WatchSource:0}: Error finding container 31bbad8d6eea61b1ac0a43d3b313aed602f2c0b9219aa91065d1573941386f0c: Status 404 returned error can't find the container with id 31bbad8d6eea61b1ac0a43d3b313aed602f2c0b9219aa91065d1573941386f0c Sep 17 19:06:35.158857 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:35.158835 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b46af12_297a_4da0_b3f3_730424604854.slice/crio-45fc66b7cced98c1fe829ff15eac0d095ff06e9d9845aef735305bf0ecc98208 WatchSource:0}: Error finding container 45fc66b7cced98c1fe829ff15eac0d095ff06e9d9845aef735305bf0ecc98208: Status 404 returned error can't find the container with id 45fc66b7cced98c1fe829ff15eac0d095ff06e9d9845aef735305bf0ecc98208 Sep 17 19:06:35.160921 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:35.160873 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5827bc3d_1d3b_4b60_9217_ffe0998fc733.slice/crio-bded8f1614f35d0de3979425e113ffddd8219016db5a054f8ed1da9473e95d45 WatchSource:0}: Error finding container bded8f1614f35d0de3979425e113ffddd8219016db5a054f8ed1da9473e95d45: Status 404 returned error can't find the container with id bded8f1614f35d0de3979425e113ffddd8219016db5a054f8ed1da9473e95d45 Sep 17 19:06:35.168217 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:35.168194 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76086d58_ef7d_4a2c_9614_c15931bac3df.slice/crio-51e074736ee747807f3dd4a742d5f4c2d39174ba58c6fa2580d46fbd494e0820 WatchSource:0}: Error finding container 51e074736ee747807f3dd4a742d5f4c2d39174ba58c6fa2580d46fbd494e0820: Status 404 returned error can't find the container with id 51e074736ee747807f3dd4a742d5f4c2d39174ba58c6fa2580d46fbd494e0820 Sep 17 19:06:35.177493 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.177457 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 19:01:34 +0000 UTC" deadline="2028-03-26 17:05:23.499950995 +0000 UTC" Sep 17 19:06:35.177493 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.177493 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13341h58m48.322462479s" Sep 17 19:06:35.189675 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:35.189648 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc13b5773_a330_4e28_8c36_dbed2847dd86.slice/crio-994298b662926fd733b7b8eede261f90e40b2816539f4562c75717a26b662dcd WatchSource:0}: Error finding container 994298b662926fd733b7b8eede261f90e40b2816539f4562c75717a26b662dcd: Status 404 returned error can't find the container with id 994298b662926fd733b7b8eede261f90e40b2816539f4562c75717a26b662dcd Sep 17 19:06:35.190760 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:06:35.190739 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58883cfb_501e_4277_9ece_f19ed6452fb7.slice/crio-6c88ea6f7d168b43c7df2038fbed22e638e46a94b8ce140843bcae24f7244cbc WatchSource:0}: Error finding container 6c88ea6f7d168b43c7df2038fbed22e638e46a94b8ce140843bcae24f7244cbc: Status 404 returned error can't find the container with id 6c88ea6f7d168b43c7df2038fbed22e638e46a94b8ce140843bcae24f7244cbc Sep 17 19:06:35.229609 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.229563 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:35.229744 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:35.229683 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:35.237405 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.237372 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" event={"ID":"8b46af12-297a-4da0-b3f3-730424604854","Type":"ContainerStarted","Data":"45fc66b7cced98c1fe829ff15eac0d095ff06e9d9845aef735305bf0ecc98208"} Sep 17 19:06:35.238702 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.238677 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q89g2" event={"ID":"76086d58-ef7d-4a2c-9614-c15931bac3df","Type":"ContainerStarted","Data":"51e074736ee747807f3dd4a742d5f4c2d39174ba58c6fa2580d46fbd494e0820"} Sep 17 19:06:35.240058 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.240034 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r5nkm" event={"ID":"5ce6506b-9752-4fbe-9622-29ecf358b465","Type":"ContainerStarted","Data":"9d4566024f756a5b93fb94b4698bf9f9e03d1d2abe26180429c16dde57c42275"} Sep 17 19:06:35.242022 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.241987 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-69wbb" event={"ID":"5827bc3d-1d3b-4b60-9217-ffe0998fc733","Type":"ContainerStarted","Data":"bded8f1614f35d0de3979425e113ffddd8219016db5a054f8ed1da9473e95d45"} Sep 17 19:06:35.243034 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.243013 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" event={"ID":"532a9820-a0c8-4dec-a697-9e7d15057df6","Type":"ContainerStarted","Data":"cc6f4e3c486a47c1877a9724343fb847e3832e13b0dee9bb8bde340c9bbadcd6"} Sep 17 19:06:35.244213 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.244190 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-94j6j" event={"ID":"58883cfb-501e-4277-9ece-f19ed6452fb7","Type":"ContainerStarted","Data":"6c88ea6f7d168b43c7df2038fbed22e638e46a94b8ce140843bcae24f7244cbc"} Sep 17 19:06:35.245333 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.245311 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l6h2d" event={"ID":"ac28db00-3290-4731-ab0c-9963ef1d6560","Type":"ContainerStarted","Data":"31bbad8d6eea61b1ac0a43d3b313aed602f2c0b9219aa91065d1573941386f0c"} Sep 17 19:06:35.246634 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.246610 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" event={"ID":"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247","Type":"ContainerStarted","Data":"1c5e34b0a869ebd24339f24714639e7fd2726136aec81903cc8dbacc66f83894"} Sep 17 19:06:35.247745 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.247726 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" event={"ID":"c13b5773-a330-4e28-8c36-dbed2847dd86","Type":"ContainerStarted","Data":"994298b662926fd733b7b8eede261f90e40b2816539f4562c75717a26b662dcd"} Sep 17 19:06:35.725542 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.725444 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:35.725710 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:35.725614 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:35.725710 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:35.725704 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret podName:7d66c218-c6ed-4ca6-a49a-47bdc94fef2d nodeName:}" failed. No retries permitted until 2026-09-17 19:06:37.725685925 +0000 UTC m=+5.987559330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret") pod "global-pull-secret-syncer-xvjpg" (UID: "7d66c218-c6ed-4ca6-a49a-47bdc94fef2d") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:35.826159 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.826119 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:35.826340 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:35.826265 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:35.826340 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:35.826331 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs podName:431387c9-5a3c-44d5-bad2-94a08bc73700 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:37.826317849 +0000 UTC m=+6.088191248 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs") pod "network-metrics-daemon-6mspz" (UID: "431387c9-5a3c-44d5-bad2-94a08bc73700") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:35.926534 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:35.926494 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hx9f\" (UniqueName: \"kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f\") pod \"network-check-target-z7q6d\" (UID: \"357e149f-654a-4188-9229-1a440c1c30a9\") " pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:35.926761 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:35.926738 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:35.926855 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:35.926768 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:35.926855 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:35.926781 2563 projected.go:196] Error preparing data for projected volume kube-api-access-2hx9f for pod openshift-network-diagnostics/network-check-target-z7q6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:35.926855 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:35.926846 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f podName:357e149f-654a-4188-9229-1a440c1c30a9 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:37.926821983 +0000 UTC m=+6.188695373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2hx9f" (UniqueName: "kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f") pod "network-check-target-z7q6d" (UID: "357e149f-654a-4188-9229-1a440c1c30a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:36.178367 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:36.178261 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 19:01:34 +0000 UTC" deadline="2028-04-18 07:54:45.48162808 +0000 UTC" Sep 17 19:06:36.178367 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:36.178307 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13884h48m9.30332683s" Sep 17 19:06:36.231526 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:36.231495 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:36.232018 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:36.231980 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:36.242306 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:36.242256 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:36.242464 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:36.242435 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:36.264613 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:36.264189 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" event={"ID":"aac0a2a35937da63c2df9c101e366883","Type":"ContainerStarted","Data":"08fcfc0218e1701c184c8341f149832a1c06af5abf5f17a9faed4ff203db8914"} Sep 17 19:06:36.271266 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:36.271016 2563 generic.go:356] "Generic (PLEG): container finished" podID="4563741808a577ad9736e2cd839b878e" containerID="c5bdea29f28b7c986cc745c27093f58ddcac51575bd719e893ae181ddab542d9" exitCode=0 Sep 17 19:06:36.271266 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:36.271062 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" event={"ID":"4563741808a577ad9736e2cd839b878e","Type":"ContainerDied","Data":"c5bdea29f28b7c986cc745c27093f58ddcac51575bd719e893ae181ddab542d9"} Sep 17 19:06:36.297702 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:36.297157 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" podStartSLOduration=3.297136865 podStartE2EDuration="3.297136865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:36.280863981 +0000 UTC m=+4.542737396" watchObservedRunningTime="2026-09-17 19:06:36.297136865 +0000 UTC m=+4.559010273" Sep 17 19:06:37.229741 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:37.228967 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:37.229741 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:37.229363 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:37.294212 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:37.294177 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" event={"ID":"4563741808a577ad9736e2cd839b878e","Type":"ContainerStarted","Data":"9f36e6e84201416d61cd2da9eef53acf478e10d3e8e7217a5903bbf6b7c2759b"} Sep 17 19:06:37.323825 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:37.323721 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" podStartSLOduration=4.323699106 podStartE2EDuration="4.323699106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:37.323035861 +0000 UTC m=+5.584909308" watchObservedRunningTime="2026-09-17 19:06:37.323699106 +0000 UTC m=+5.585572514" Sep 17 19:06:37.748118 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:37.744944 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:37.748118 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:37.745140 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:37.748118 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:37.745208 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret podName:7d66c218-c6ed-4ca6-a49a-47bdc94fef2d nodeName:}" failed. No retries permitted until 2026-09-17 19:06:41.745190229 +0000 UTC m=+10.007063636 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret") pod "global-pull-secret-syncer-xvjpg" (UID: "7d66c218-c6ed-4ca6-a49a-47bdc94fef2d") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:37.846742 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:37.846115 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:37.846742 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:37.846325 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:37.846742 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:37.846392 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs podName:431387c9-5a3c-44d5-bad2-94a08bc73700 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:41.846373665 +0000 UTC m=+10.108247056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs") pod "network-metrics-daemon-6mspz" (UID: "431387c9-5a3c-44d5-bad2-94a08bc73700") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:37.947640 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:37.946915 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hx9f\" (UniqueName: \"kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f\") pod \"network-check-target-z7q6d\" (UID: \"357e149f-654a-4188-9229-1a440c1c30a9\") " pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:37.947640 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:37.947120 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:37.947640 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:37.947144 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:37.947640 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:37.947157 2563 projected.go:196] Error preparing data for projected volume kube-api-access-2hx9f for pod openshift-network-diagnostics/network-check-target-z7q6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:37.947640 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:37.947261 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f podName:357e149f-654a-4188-9229-1a440c1c30a9 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:41.947242695 +0000 UTC m=+10.209116084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2hx9f" (UniqueName: "kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f") pod "network-check-target-z7q6d" (UID: "357e149f-654a-4188-9229-1a440c1c30a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:38.229796 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:38.229244 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:38.229796 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:38.229385 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:38.230367 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:38.229840 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:38.230367 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:38.229938 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:39.229357 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:39.229324 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:39.229565 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:39.229536 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:40.231037 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:40.229620 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:40.231037 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:40.229760 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:40.231037 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:40.230311 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:40.231037 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:40.230423 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:41.229414 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:41.229292 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:41.229582 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:41.229429 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:41.780300 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:41.780260 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:41.780781 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:41.780398 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:41.780781 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:41.780446 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret podName:7d66c218-c6ed-4ca6-a49a-47bdc94fef2d nodeName:}" failed. No retries permitted until 2026-09-17 19:06:49.780433478 +0000 UTC m=+18.042306877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret") pod "global-pull-secret-syncer-xvjpg" (UID: "7d66c218-c6ed-4ca6-a49a-47bdc94fef2d") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:41.881409 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:41.881371 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:41.881597 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:41.881505 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:41.881597 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:41.881567 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs podName:431387c9-5a3c-44d5-bad2-94a08bc73700 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:49.881549158 +0000 UTC m=+18.143422559 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs") pod "network-metrics-daemon-6mspz" (UID: "431387c9-5a3c-44d5-bad2-94a08bc73700") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:41.981893 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:41.981858 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hx9f\" (UniqueName: \"kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f\") pod \"network-check-target-z7q6d\" (UID: \"357e149f-654a-4188-9229-1a440c1c30a9\") " pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:41.982075 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:41.982055 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:41.982135 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:41.982082 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:41.982135 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:41.982111 2563 projected.go:196] Error preparing data for projected volume kube-api-access-2hx9f for pod openshift-network-diagnostics/network-check-target-z7q6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:41.982214 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:41.982169 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f podName:357e149f-654a-4188-9229-1a440c1c30a9 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:49.982154124 +0000 UTC m=+18.244027509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2hx9f" (UniqueName: "kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f") pod "network-check-target-z7q6d" (UID: "357e149f-654a-4188-9229-1a440c1c30a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:42.229916 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:42.229834 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:42.230089 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:42.229919 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:42.231030 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:42.230336 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:42.231030 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:42.230464 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:43.228776 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:43.228738 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:43.229261 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:43.228882 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:44.230917 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:44.230878 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:44.231343 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:44.231009 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:44.231450 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:44.231429 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:44.231554 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:44.231536 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:45.228628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:45.228595 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:45.228795 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:45.228713 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:46.228707 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:46.228668 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:46.229240 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:46.228669 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:46.229240 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:46.228806 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:46.229240 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:46.228931 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:47.229480 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:47.229437 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:47.229936 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:47.229573 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:48.229290 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:48.229253 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:48.229488 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:48.229391 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:48.229488 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:48.229430 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:48.229823 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:48.229517 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:49.229415 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:49.229363 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:49.229667 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:49.229509 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:49.845245 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:49.845205 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:49.845421 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:49.845376 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:49.845498 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:49.845452 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret podName:7d66c218-c6ed-4ca6-a49a-47bdc94fef2d nodeName:}" failed. No retries permitted until 2026-09-17 19:07:05.845430968 +0000 UTC m=+34.107304379 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret") pod "global-pull-secret-syncer-xvjpg" (UID: "7d66c218-c6ed-4ca6-a49a-47bdc94fef2d") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:49.946263 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:49.946223 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:49.946434 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:49.946340 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:49.946434 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:49.946409 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs podName:431387c9-5a3c-44d5-bad2-94a08bc73700 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:05.946392233 +0000 UTC m=+34.208265620 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs") pod "network-metrics-daemon-6mspz" (UID: "431387c9-5a3c-44d5-bad2-94a08bc73700") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:50.046876 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:50.046839 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hx9f\" (UniqueName: \"kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f\") pod \"network-check-target-z7q6d\" (UID: \"357e149f-654a-4188-9229-1a440c1c30a9\") " pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:50.047064 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:50.047018 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:50.047064 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:50.047040 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:50.047064 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:50.047050 2563 projected.go:196] Error preparing data for projected volume kube-api-access-2hx9f for pod openshift-network-diagnostics/network-check-target-z7q6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:50.047245 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:50.047126 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f podName:357e149f-654a-4188-9229-1a440c1c30a9 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:06.047091935 +0000 UTC m=+34.308965327 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2hx9f" (UniqueName: "kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f") pod "network-check-target-z7q6d" (UID: "357e149f-654a-4188-9229-1a440c1c30a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:50.229111 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:50.229012 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:50.229111 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:50.229028 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:50.229349 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:50.229161 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:50.229349 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:50.229304 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:51.229252 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:51.229221 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:51.229672 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:51.229328 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:52.230476 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:52.230435 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:52.230932 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:52.230558 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:52.230932 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:52.230913 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:52.231055 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:52.231034 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:53.228915 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.228755 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:53.229017 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:53.228991 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:53.329751 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.329721 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" event={"ID":"c13b5773-a330-4e28-8c36-dbed2847dd86","Type":"ContainerStarted","Data":"3d55716b1a826763529e247e290a593127db0bd17bd10322fb0d5a3448068ba0"} Sep 17 19:06:53.330504 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.329795 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" event={"ID":"c13b5773-a330-4e28-8c36-dbed2847dd86","Type":"ContainerStarted","Data":"e28ed43236523357e9484a8fa4c80c08583bf13bb784ec0cd1d3d38b96bac17b"} Sep 17 19:06:53.330504 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.329831 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" event={"ID":"c13b5773-a330-4e28-8c36-dbed2847dd86","Type":"ContainerStarted","Data":"c78c804e0f811d560c5ce152d3a33bcad13cd405838b0875877519a8e538de9a"} Sep 17 19:06:53.330504 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.329859 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" event={"ID":"c13b5773-a330-4e28-8c36-dbed2847dd86","Type":"ContainerStarted","Data":"8815a1900d0a98cd4ad9542bbf6bf033c3ec428fba1daca06bb752744b993ab1"} Sep 17 19:06:53.331350 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.331324 2563 generic.go:356] "Generic (PLEG): container finished" podID="8b46af12-297a-4da0-b3f3-730424604854" containerID="2d83035a953837022955207afabe3849e4b21208c4ddc2286255139a25f3b0c6" exitCode=0 Sep 17 19:06:53.331460 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.331419 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" event={"ID":"8b46af12-297a-4da0-b3f3-730424604854","Type":"ContainerDied","Data":"2d83035a953837022955207afabe3849e4b21208c4ddc2286255139a25f3b0c6"} Sep 17 19:06:53.332958 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.332928 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q89g2" event={"ID":"76086d58-ef7d-4a2c-9614-c15931bac3df","Type":"ContainerStarted","Data":"f594f031d62b0a59c52e1da1904a3e119f94e2fca0f8042b544474ef12245bb6"} Sep 17 19:06:53.334484 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.334463 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r5nkm" event={"ID":"5ce6506b-9752-4fbe-9622-29ecf358b465","Type":"ContainerStarted","Data":"7783e18e4ad286cdf60d857ccb1bc040ba29a126729fae099a16161357f39abd"} Sep 17 19:06:53.336058 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.336034 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" event={"ID":"532a9820-a0c8-4dec-a697-9e7d15057df6","Type":"ContainerStarted","Data":"f29a4c104c0d9c5b3eb963a9bd628dd0ac5597209df099fb8d5d359f232158f2"} Sep 17 19:06:53.337472 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.337450 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-94j6j" event={"ID":"58883cfb-501e-4277-9ece-f19ed6452fb7","Type":"ContainerStarted","Data":"944ab2681bba438a4dd4793e2bb9847efc4046a9785c7bb35ce9c63d9e22111d"} Sep 17 19:06:53.339061 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.339039 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l6h2d" event={"ID":"ac28db00-3290-4731-ab0c-9963ef1d6560","Type":"ContainerStarted","Data":"590379a7e596ee1e36f3c0a7fb78e803d07b98eb7e05e7aed2f413c7f0139292"} Sep 17 19:06:53.340429 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.340414 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" event={"ID":"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247","Type":"ContainerStarted","Data":"abecfaceb459e91f1fb47c4247f20f9b3a1a5555fbcf3707ba0af306a2fc0cd3"} Sep 17 19:06:53.386403 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.386304 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l6h2d" podStartSLOduration=3.681804118 podStartE2EDuration="21.386288517s" totalImagesPullingTime="17.704484399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:53.38463212 +0000 UTC m=+21.646505541" watchObservedRunningTime="2026-09-17 19:06:53.386288517 +0000 UTC m=+21.648161924" Sep 17 19:06:53.415609 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.415563 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dc7vl" podStartSLOduration=3.702041349 podStartE2EDuration="21.415548026s" totalImagesPullingTime="17.713506677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:53.414711746 +0000 UTC m=+21.676585154" watchObservedRunningTime="2026-09-17 19:06:53.415548026 +0000 UTC m=+21.677421431" Sep 17 19:06:53.470286 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.470239 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-94j6j" podStartSLOduration=7.360667818 podStartE2EDuration="21.470219848s" totalImagesPullingTime="14.10955203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:53.469590432 +0000 UTC m=+21.731463858" watchObservedRunningTime="2026-09-17 19:06:53.470219848 +0000 UTC m=+21.732093256" Sep 17 19:06:53.470405 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:53.470392 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-r5nkm" podStartSLOduration=3.734675841 podStartE2EDuration="21.470385422s" totalImagesPullingTime="17.735709581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:53.445248936 +0000 UTC m=+21.707122362" watchObservedRunningTime="2026-09-17 19:06:53.470385422 +0000 UTC m=+21.732258828" Sep 17 19:06:54.229016 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:54.228982 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:54.229219 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:54.229157 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:54.229219 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:54.229181 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:54.229354 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:54.229339 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:54.346445 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:54.346404 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" event={"ID":"c13b5773-a330-4e28-8c36-dbed2847dd86","Type":"ContainerStarted","Data":"b5a59ed572d7b119597af765593d948cdcb7d19463a8f234205e49983b8882f0"} Sep 17 19:06:54.347012 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:54.346484 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" event={"ID":"c13b5773-a330-4e28-8c36-dbed2847dd86","Type":"ContainerStarted","Data":"1b04dd9b7c43715bd56d080c7d702506660408d1281e3b62c51617f2d79eb461"} Sep 17 19:06:54.348493 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:54.348441 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-69wbb" event={"ID":"5827bc3d-1d3b-4b60-9217-ffe0998fc733","Type":"ContainerStarted","Data":"ca360ee613cd1a1c0950ff8c57ea9fb7af2c1577eb9d05d5784889648ed2f88e"} Sep 17 19:06:54.371600 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:54.371550 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-q89g2" podStartSLOduration=4.692884014 podStartE2EDuration="22.37152993s" totalImagesPullingTime="17.678645916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:53.500925949 +0000 UTC m=+21.762799356" watchObservedRunningTime="2026-09-17 19:06:54.37152993 +0000 UTC m=+22.633403372" Sep 17 19:06:54.565819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:54.565636 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 17 19:06:55.168873 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:55.168737 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T19:06:54.565808626Z","UUID":"c187042a-a0ff-4fb5-bdf9-59b841a4342b","Handler":null,"Name":"","Endpoint":""} Sep 17 19:06:55.171089 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:55.171064 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 17 19:06:55.171257 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:55.171130 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 17 19:06:55.229198 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:55.229165 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:55.229367 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:55.229288 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:55.353235 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:55.353202 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" event={"ID":"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247","Type":"ContainerStarted","Data":"709b594e55bcbfc5c3b4a71faf9394a31c112e24ce5b4a5ec3cf511d62437e42"} Sep 17 19:06:56.229199 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:56.229162 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:56.229385 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:56.229167 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:56.229385 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:56.229305 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:56.229483 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:56.229387 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:56.359706 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:56.359666 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" event={"ID":"c13b5773-a330-4e28-8c36-dbed2847dd86","Type":"ContainerStarted","Data":"410de8b7230f60c236607c58cd74cc78f87362639f01ce984e92e0800ed3387c"} Sep 17 19:06:56.362167 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:56.362137 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" event={"ID":"6f7ec5b3-e475-40c6-b4eb-97a3fe2ef247","Type":"ContainerStarted","Data":"5f9d4b7b51b65e08a420dac17f1ce9d22f2f96acfd492f6dc8dee9db62e5daca"} Sep 17 19:06:56.391068 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:56.391018 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-69wbb" podStartSLOduration=10.255233577 podStartE2EDuration="24.391001965s" totalImagesPullingTime="14.135768388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:54.372174598 +0000 UTC m=+22.634048024" watchObservedRunningTime="2026-09-17 19:06:56.391001965 +0000 UTC m=+24.652875371" Sep 17 19:06:57.209376 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:57.209339 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:06:57.210025 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:57.210001 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:06:57.227817 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:57.227758 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prjm8" podStartSLOduration=5.183921511 podStartE2EDuration="25.227740362s" totalImagesPullingTime="20.043818851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:56.390958259 +0000 UTC m=+24.652831688" watchObservedRunningTime="2026-09-17 19:06:57.227740362 +0000 UTC m=+25.489613768" Sep 17 19:06:57.228825 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:57.228803 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:57.228938 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:57.228918 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:06:58.229365 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:58.229177 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:06:58.229798 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:58.229177 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:06:58.229798 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:58.229474 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:06:58.229798 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:58.229516 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:06:58.370689 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:58.370593 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" event={"ID":"c13b5773-a330-4e28-8c36-dbed2847dd86","Type":"ContainerStarted","Data":"ec451ad2990f3395feee7bb0afd39ca3a9a538ebd67e50565ee7cc217c92ccfa"} Sep 17 19:06:58.370962 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:58.370934 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:58.371060 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:58.371008 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:58.371060 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:58.371053 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:58.372866 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:58.372837 2563 generic.go:356] "Generic (PLEG): container finished" podID="8b46af12-297a-4da0-b3f3-730424604854" containerID="e0f5a3de46ecab493ef8d40b2f5386474d07abfd442932d6b35bf51d3191965e" exitCode=0 Sep 17 19:06:58.372986 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:58.372899 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" event={"ID":"8b46af12-297a-4da0-b3f3-730424604854","Type":"ContainerDied","Data":"e0f5a3de46ecab493ef8d40b2f5386474d07abfd442932d6b35bf51d3191965e"} Sep 17 19:06:58.387322 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:58.387294 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:58.387676 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:58.387661 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:06:58.437370 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:58.437317 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" podStartSLOduration=8.692319655 podStartE2EDuration="26.437301201s" totalImagesPullingTime="17.744981546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:58.408340797 +0000 UTC m=+26.670214204" watchObservedRunningTime="2026-09-17 19:06:58.437301201 +0000 UTC m=+26.699174608" Sep 17 19:06:59.229220 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:06:59.229165 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:06:59.229456 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:06:59.229288 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:07:00.153119 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:00.153064 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xvjpg"] Sep 17 19:07:00.153272 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:00.153252 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:07:00.153360 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:00.153338 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:07:00.156844 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:00.156817 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6mspz"] Sep 17 19:07:00.157016 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:00.156946 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:07:00.157082 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:00.157063 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:07:00.157474 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:00.157454 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7q6d"] Sep 17 19:07:00.157555 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:00.157543 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:07:00.157635 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:00.157618 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:07:00.379271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:00.379239 2563 generic.go:356] "Generic (PLEG): container finished" podID="8b46af12-297a-4da0-b3f3-730424604854" containerID="7b1f316259a685c98019c87e5c81b6362abe33c3ef45c86a287b0ceaff9e4853" exitCode=0 Sep 17 19:07:00.379844 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:00.379331 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" event={"ID":"8b46af12-297a-4da0-b3f3-730424604854","Type":"ContainerDied","Data":"7b1f316259a685c98019c87e5c81b6362abe33c3ef45c86a287b0ceaff9e4853"} Sep 17 19:07:00.551343 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:00.551260 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:07:00.551492 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:00.551399 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 17 19:07:00.552023 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:00.552003 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-94j6j" Sep 17 19:07:02.231981 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:02.231016 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:07:02.231981 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:02.231181 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:07:02.231981 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:02.231623 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:07:02.231981 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:02.231727 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:07:02.231981 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:02.231795 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:07:02.231981 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:02.231957 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:07:02.385443 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:02.385407 2563 generic.go:356] "Generic (PLEG): container finished" podID="8b46af12-297a-4da0-b3f3-730424604854" containerID="56c6fdcd4a98f9da16f6e594af5cfab01d983dc613969d07b19ed1f2c4f79e71" exitCode=0 Sep 17 19:07:02.385569 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:02.385463 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" event={"ID":"8b46af12-297a-4da0-b3f3-730424604854","Type":"ContainerDied","Data":"56c6fdcd4a98f9da16f6e594af5cfab01d983dc613969d07b19ed1f2c4f79e71"} Sep 17 19:07:04.229197 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:04.228912 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:07:04.229678 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:04.228912 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:07:04.229678 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:04.229306 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvjpg" podUID="7d66c218-c6ed-4ca6-a49a-47bdc94fef2d" Sep 17 19:07:04.229678 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:04.229328 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7q6d" podUID="357e149f-654a-4188-9229-1a440c1c30a9" Sep 17 19:07:04.229678 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:04.228962 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:07:04.229678 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:04.229413 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6mspz" podUID="431387c9-5a3c-44d5-bad2-94a08bc73700" Sep 17 19:07:05.453249 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.453223 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeReady" Sep 17 19:07:05.453644 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.453402 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 17 19:07:05.501937 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.501900 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-758b8c48b6-7p8j9"] Sep 17 19:07:05.506481 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.504906 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.507806 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.507788 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-msk9c\"" Sep 17 19:07:05.508008 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.507973 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 17 19:07:05.508281 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.508261 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 17 19:07:05.508364 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.508323 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 17 19:07:05.515963 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.515947 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 17 19:07:05.522447 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.522424 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-758b8c48b6-7p8j9"] Sep 17 19:07:05.524978 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.524958 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fbrxt"] Sep 17 19:07:05.526983 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.526866 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.530032 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.529988 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 17 19:07:05.530032 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.530021 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 17 19:07:05.530576 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.530560 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dpmzt\"" Sep 17 19:07:05.530752 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.530732 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qlb49"] Sep 17 19:07:05.533582 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.533564 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qlb49" Sep 17 19:07:05.535941 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.535925 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 17 19:07:05.536022 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.535984 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 17 19:07:05.536288 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.536270 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 17 19:07:05.536638 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.536622 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4xslp\"" Sep 17 19:07:05.546167 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.546142 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fbrxt"] Sep 17 19:07:05.547665 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.547642 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qlb49"] Sep 17 19:07:05.656913 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.656873 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-bound-sa-token\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.656913 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.656908 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/826c88c0-6862-4bc7-ad50-51c7365ded8b-cert\") pod \"ingress-canary-qlb49\" (UID: \"826c88c0-6862-4bc7-ad50-51c7365ded8b\") " pod="openshift-ingress-canary/ingress-canary-qlb49" Sep 17 19:07:05.657137 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.656945 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-image-registry-private-configuration\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.657137 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.657038 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j77jt\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-kube-api-access-j77jt\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.657137 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.657066 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/92e91b8e-d907-451f-8653-29bd65f28caa-config-volume\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.657137 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.657086 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2b58566f-f608-4e4d-9e2f-4bd88bb65202-ca-trust-extracted\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.657137 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.657115 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-certificates\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.657137 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.657132 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-installation-pull-secrets\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.657359 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.657159 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.657359 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.657188 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mxsr\" (UniqueName: \"kubernetes.io/projected/826c88c0-6862-4bc7-ad50-51c7365ded8b-kube-api-access-9mxsr\") pod \"ingress-canary-qlb49\" (UID: \"826c88c0-6862-4bc7-ad50-51c7365ded8b\") " pod="openshift-ingress-canary/ingress-canary-qlb49" Sep 17 19:07:05.657359 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.657218 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.657359 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.657240 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8vsz\" (UniqueName: \"kubernetes.io/projected/92e91b8e-d907-451f-8653-29bd65f28caa-kube-api-access-c8vsz\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.657359 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.657265 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-trusted-ca\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.657359 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.657295 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e91b8e-d907-451f-8653-29bd65f28caa-tmp-dir\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.758295 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758198 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-bound-sa-token\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.758539 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758336 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/826c88c0-6862-4bc7-ad50-51c7365ded8b-cert\") pod \"ingress-canary-qlb49\" (UID: \"826c88c0-6862-4bc7-ad50-51c7365ded8b\") " pod="openshift-ingress-canary/ingress-canary-qlb49" Sep 17 19:07:05.758539 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758370 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-image-registry-private-configuration\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.758539 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758501 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j77jt\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-kube-api-access-j77jt\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.758757 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758558 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/92e91b8e-d907-451f-8653-29bd65f28caa-config-volume\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.758757 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758596 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2b58566f-f608-4e4d-9e2f-4bd88bb65202-ca-trust-extracted\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.758757 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758621 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-certificates\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.758757 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758645 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-installation-pull-secrets\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.758757 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758673 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.758757 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758706 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mxsr\" (UniqueName: \"kubernetes.io/projected/826c88c0-6862-4bc7-ad50-51c7365ded8b-kube-api-access-9mxsr\") pod \"ingress-canary-qlb49\" (UID: \"826c88c0-6862-4bc7-ad50-51c7365ded8b\") " pod="openshift-ingress-canary/ingress-canary-qlb49" Sep 17 19:07:05.758757 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758741 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.759075 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758776 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8vsz\" (UniqueName: \"kubernetes.io/projected/92e91b8e-d907-451f-8653-29bd65f28caa-kube-api-access-c8vsz\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.759075 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758814 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-trusted-ca\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.759075 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.758859 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e91b8e-d907-451f-8653-29bd65f28caa-tmp-dir\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.759255 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.759128 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2b58566f-f608-4e4d-9e2f-4bd88bb65202-ca-trust-extracted\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.759255 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.759179 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/92e91b8e-d907-451f-8653-29bd65f28caa-config-volume\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.759255 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.759212 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e91b8e-d907-451f-8653-29bd65f28caa-tmp-dir\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.759405 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:05.759215 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:05.759405 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:05.759333 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls podName:92e91b8e-d907-451f-8653-29bd65f28caa nodeName:}" failed. No retries permitted until 2026-09-17 19:07:06.259319746 +0000 UTC m=+34.521193131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls") pod "dns-default-fbrxt" (UID: "92e91b8e-d907-451f-8653-29bd65f28caa") : secret "dns-default-metrics-tls" not found Sep 17 19:07:05.759405 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:05.759272 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 19:07:05.759405 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:05.759351 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758b8c48b6-7p8j9: secret "image-registry-tls" not found Sep 17 19:07:05.759405 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:05.759371 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls podName:2b58566f-f608-4e4d-9e2f-4bd88bb65202 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:06.259365197 +0000 UTC m=+34.521238581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls") pod "image-registry-758b8c48b6-7p8j9" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202") : secret "image-registry-tls" not found Sep 17 19:07:05.759628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.759520 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-certificates\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.760071 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.760048 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-trusted-ca\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.763065 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.763040 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-image-registry-private-configuration\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.763187 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.763062 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/826c88c0-6862-4bc7-ad50-51c7365ded8b-cert\") pod \"ingress-canary-qlb49\" (UID: \"826c88c0-6862-4bc7-ad50-51c7365ded8b\") " pod="openshift-ingress-canary/ingress-canary-qlb49" Sep 17 19:07:05.764494 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.764456 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-installation-pull-secrets\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.770708 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.770686 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-bound-sa-token\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.770947 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.770928 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j77jt\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-kube-api-access-j77jt\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:05.773824 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.773798 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8vsz\" (UniqueName: \"kubernetes.io/projected/92e91b8e-d907-451f-8653-29bd65f28caa-kube-api-access-c8vsz\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:05.774069 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.774052 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mxsr\" (UniqueName: \"kubernetes.io/projected/826c88c0-6862-4bc7-ad50-51c7365ded8b-kube-api-access-9mxsr\") pod \"ingress-canary-qlb49\" (UID: \"826c88c0-6862-4bc7-ad50-51c7365ded8b\") " pod="openshift-ingress-canary/ingress-canary-qlb49" Sep 17 19:07:05.844236 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.844146 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qlb49" Sep 17 19:07:05.859773 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.859746 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:07:05.859887 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:05.859867 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:07:05.859951 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:05.859923 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret podName:7d66c218-c6ed-4ca6-a49a-47bdc94fef2d nodeName:}" failed. No retries permitted until 2026-09-17 19:07:37.859911109 +0000 UTC m=+66.121784506 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret") pod "global-pull-secret-syncer-xvjpg" (UID: "7d66c218-c6ed-4ca6-a49a-47bdc94fef2d") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:07:05.960460 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:05.960420 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:07:05.960624 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:05.960578 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:07:05.960666 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:05.960643 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs podName:431387c9-5a3c-44d5-bad2-94a08bc73700 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:37.960628145 +0000 UTC m=+66.222501542 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs") pod "network-metrics-daemon-6mspz" (UID: "431387c9-5a3c-44d5-bad2-94a08bc73700") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:07:06.061773 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.061736 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hx9f\" (UniqueName: \"kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f\") pod \"network-check-target-z7q6d\" (UID: \"357e149f-654a-4188-9229-1a440c1c30a9\") " pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:07:06.061933 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:06.061904 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:07:06.062037 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:06.061937 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:07:06.062037 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:06.061952 2563 projected.go:196] Error preparing data for projected volume kube-api-access-2hx9f for pod openshift-network-diagnostics/network-check-target-z7q6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:07:06.062153 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:06.062056 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f podName:357e149f-654a-4188-9229-1a440c1c30a9 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:38.062038143 +0000 UTC m=+66.323911552 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2hx9f" (UniqueName: "kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f") pod "network-check-target-z7q6d" (UID: "357e149f-654a-4188-9229-1a440c1c30a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:07:06.229662 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.229587 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:07:06.229821 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.229587 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:07:06.229821 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.229794 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:07:06.234673 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.234457 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 19:07:06.234673 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.234506 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 19:07:06.234673 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.234506 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b9ftw\"" Sep 17 19:07:06.234673 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.234590 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 19:07:06.234673 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.234598 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-47zq2\"" Sep 17 19:07:06.234673 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.234678 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 19:07:06.262807 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.262781 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:06.262927 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.262814 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:06.262927 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:06.262912 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 19:07:06.262927 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:06.262915 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:06.262927 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:06.262922 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758b8c48b6-7p8j9: secret "image-registry-tls" not found Sep 17 19:07:06.263075 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:06.262972 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls podName:2b58566f-f608-4e4d-9e2f-4bd88bb65202 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:07.262959286 +0000 UTC m=+35.524832683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls") pod "image-registry-758b8c48b6-7p8j9" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202") : secret "image-registry-tls" not found Sep 17 19:07:06.263075 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:06.262984 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls podName:92e91b8e-d907-451f-8653-29bd65f28caa nodeName:}" failed. No retries permitted until 2026-09-17 19:07:07.262978008 +0000 UTC m=+35.524851394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls") pod "dns-default-fbrxt" (UID: "92e91b8e-d907-451f-8653-29bd65f28caa") : secret "dns-default-metrics-tls" not found Sep 17 19:07:06.399707 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.399526 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" event={"ID":"8b46af12-297a-4da0-b3f3-730424604854","Type":"ContainerStarted","Data":"af9b7e7d76b0ba76485989cc63d979552030fc1e8036c57f22489288d84a9f4a"} Sep 17 19:07:06.509545 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:06.509482 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qlb49"] Sep 17 19:07:06.516951 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:06.516925 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod826c88c0_6862_4bc7_ad50_51c7365ded8b.slice/crio-95da4e7748433c83838c1fabe2344079841777c8dddf90e0a6d455521a6b6953 WatchSource:0}: Error finding container 95da4e7748433c83838c1fabe2344079841777c8dddf90e0a6d455521a6b6953: Status 404 returned error can't find the container with id 95da4e7748433c83838c1fabe2344079841777c8dddf90e0a6d455521a6b6953 Sep 17 19:07:07.270375 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.270332 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:07.270581 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:07.270461 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 19:07:07.270581 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:07.270482 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758b8c48b6-7p8j9: secret "image-registry-tls" not found Sep 17 19:07:07.270581 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.270525 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:07.270581 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:07.270546 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls podName:2b58566f-f608-4e4d-9e2f-4bd88bb65202 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:09.270524941 +0000 UTC m=+37.532398346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls") pod "image-registry-758b8c48b6-7p8j9" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202") : secret "image-registry-tls" not found Sep 17 19:07:07.270783 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:07.270619 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:07.270783 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:07.270674 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls podName:92e91b8e-d907-451f-8653-29bd65f28caa nodeName:}" failed. No retries permitted until 2026-09-17 19:07:09.270658953 +0000 UTC m=+37.532532339 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls") pod "dns-default-fbrxt" (UID: "92e91b8e-d907-451f-8653-29bd65f28caa") : secret "dns-default-metrics-tls" not found Sep 17 19:07:07.405475 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.405440 2563 generic.go:356] "Generic (PLEG): container finished" podID="8b46af12-297a-4da0-b3f3-730424604854" containerID="af9b7e7d76b0ba76485989cc63d979552030fc1e8036c57f22489288d84a9f4a" exitCode=0 Sep 17 19:07:07.405648 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.405522 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" event={"ID":"8b46af12-297a-4da0-b3f3-730424604854","Type":"ContainerDied","Data":"af9b7e7d76b0ba76485989cc63d979552030fc1e8036c57f22489288d84a9f4a"} Sep 17 19:07:07.406861 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.406843 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qlb49" event={"ID":"826c88c0-6862-4bc7-ad50-51c7365ded8b","Type":"ContainerStarted","Data":"95da4e7748433c83838c1fabe2344079841777c8dddf90e0a6d455521a6b6953"} Sep 17 19:07:07.681960 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.681928 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-mml6g"] Sep 17 19:07:07.683880 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.683857 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.686483 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.686461 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 17 19:07:07.686661 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.686608 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 17 19:07:07.686884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.686862 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-8nszr\"" Sep 17 19:07:07.687275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.687159 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 17 19:07:07.687885 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.687740 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 17 19:07:07.688426 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.688406 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s"] Sep 17 19:07:07.690591 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.690572 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:07.693735 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.693709 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 17 19:07:07.694459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.694439 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 17 19:07:07.694569 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.694548 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wrp9n\"" Sep 17 19:07:07.694824 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.694512 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 17 19:07:07.695043 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.694444 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 17 19:07:07.702375 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.702350 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-mml6g"] Sep 17 19:07:07.702488 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.702434 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s"] Sep 17 19:07:07.774260 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.774181 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm"] Sep 17 19:07:07.774500 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.774476 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0280dd0f-999f-44b2-b052-4652b01c394c-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.774560 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.774532 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:07.774611 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.774600 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0280dd0f-999f-44b2-b052-4652b01c394c-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.774672 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.774624 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0280dd0f-999f-44b2-b052-4652b01c394c-serving-cert\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.774734 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.774681 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2pmp\" (UniqueName: \"kubernetes.io/projected/0280dd0f-999f-44b2-b052-4652b01c394c-kube-api-access-w2pmp\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.774734 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.774724 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0280dd0f-999f-44b2-b052-4652b01c394c-tmp\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.774878 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.774836 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q494t\" (UniqueName: \"kubernetes.io/projected/1a196995-e126-4724-b2b5-2b108862b410-kube-api-access-q494t\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:07.774878 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.774873 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0280dd0f-999f-44b2-b052-4652b01c394c-snapshots\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.776414 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.776391 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:07.777521 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.777495 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-c8pst"] Sep 17 19:07:07.779339 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.779318 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-c8pst" Sep 17 19:07:07.779524 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.779506 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 17 19:07:07.779605 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.779586 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-n9sxx\"" Sep 17 19:07:07.780351 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.780333 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 17 19:07:07.780442 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.780388 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 17 19:07:07.780442 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.780332 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 17 19:07:07.781158 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.781140 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 17 19:07:07.781688 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.781671 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 17 19:07:07.781777 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.781706 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-rc8br\"" Sep 17 19:07:07.793563 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.793522 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-c8pst"] Sep 17 19:07:07.794918 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.794897 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm"] Sep 17 19:07:07.876271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876238 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0280dd0f-999f-44b2-b052-4652b01c394c-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.876464 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:07.876464 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876312 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvv8c\" (UniqueName: \"kubernetes.io/projected/3e4c6b0f-837d-4eca-b8f4-c922e1de910d-kube-api-access-jvv8c\") pod \"volume-data-source-validator-5c4db8bdf9-c8pst\" (UID: \"3e4c6b0f-837d-4eca-b8f4-c922e1de910d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-c8pst" Sep 17 19:07:07.876464 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876355 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0280dd0f-999f-44b2-b052-4652b01c394c-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.876464 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:07.876449 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 19:07:07.876677 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:07.876559 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls podName:1a196995-e126-4724-b2b5-2b108862b410 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:08.376543551 +0000 UTC m=+36.638416936 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7nc5s" (UID: "1a196995-e126-4724-b2b5-2b108862b410") : secret "samples-operator-tls" not found Sep 17 19:07:07.876677 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876452 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0280dd0f-999f-44b2-b052-4652b01c394c-serving-cert\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.876677 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876616 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2pmp\" (UniqueName: \"kubernetes.io/projected/0280dd0f-999f-44b2-b052-4652b01c394c-kube-api-access-w2pmp\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.876677 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876645 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd2sq\" (UniqueName: \"kubernetes.io/projected/8df8584c-c2db-416a-a790-6a51db7aea0e-kube-api-access-hd2sq\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:07.876860 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876679 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0280dd0f-999f-44b2-b052-4652b01c394c-tmp\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.876860 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876739 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8df8584c-c2db-416a-a790-6a51db7aea0e-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:07.876860 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876767 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:07.876860 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876808 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q494t\" (UniqueName: \"kubernetes.io/projected/1a196995-e126-4724-b2b5-2b108862b410-kube-api-access-q494t\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:07.876860 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876836 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0280dd0f-999f-44b2-b052-4652b01c394c-snapshots\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.877134 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.876932 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0280dd0f-999f-44b2-b052-4652b01c394c-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.877134 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.877090 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0280dd0f-999f-44b2-b052-4652b01c394c-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.877385 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.877366 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0280dd0f-999f-44b2-b052-4652b01c394c-snapshots\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.877447 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.877370 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0280dd0f-999f-44b2-b052-4652b01c394c-tmp\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.879818 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.879798 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0280dd0f-999f-44b2-b052-4652b01c394c-serving-cert\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.890995 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.890970 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-544cd45fb-8wb98"] Sep 17 19:07:07.893619 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.893596 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:07.897010 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.896989 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2pmp\" (UniqueName: \"kubernetes.io/projected/0280dd0f-999f-44b2-b052-4652b01c394c-kube-api-access-w2pmp\") pod \"insights-operator-6f9c644dd9-mml6g\" (UID: \"0280dd0f-999f-44b2-b052-4652b01c394c\") " pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:07.897135 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.897016 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q494t\" (UniqueName: \"kubernetes.io/projected/1a196995-e126-4724-b2b5-2b108862b410-kube-api-access-q494t\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:07.897352 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.897333 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 17 19:07:07.897490 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.897472 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 17 19:07:07.897632 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.897617 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 17 19:07:07.897716 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.897677 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 17 19:07:07.898046 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.898029 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-65f9b\"" Sep 17 19:07:07.898154 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.898082 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 17 19:07:07.898154 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.898087 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 17 19:07:07.907999 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.907973 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-544cd45fb-8wb98"] Sep 17 19:07:07.977954 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.977916 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:07.978185 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.977968 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:07.978185 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.978017 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-stats-auth\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:07.978185 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.978058 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:07.978185 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.978165 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-default-certificate\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:07.978385 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:07.978186 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:07.978385 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:07.978262 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls podName:8df8584c-c2db-416a-a790-6a51db7aea0e nodeName:}" failed. No retries permitted until 2026-09-17 19:07:08.478243174 +0000 UTC m=+36.740116579 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-jb6mm" (UID: "8df8584c-c2db-416a-a790-6a51db7aea0e") : secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:07.978385 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.978284 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvv8c\" (UniqueName: \"kubernetes.io/projected/3e4c6b0f-837d-4eca-b8f4-c922e1de910d-kube-api-access-jvv8c\") pod \"volume-data-source-validator-5c4db8bdf9-c8pst\" (UID: \"3e4c6b0f-837d-4eca-b8f4-c922e1de910d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-c8pst" Sep 17 19:07:07.978385 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.978336 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd2sq\" (UniqueName: \"kubernetes.io/projected/8df8584c-c2db-416a-a790-6a51db7aea0e-kube-api-access-hd2sq\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:07.978385 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.978363 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpj56\" (UniqueName: \"kubernetes.io/projected/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-kube-api-access-lpj56\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:07.978616 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.978416 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8df8584c-c2db-416a-a790-6a51db7aea0e-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:07.979756 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.979735 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8df8584c-c2db-416a-a790-6a51db7aea0e-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:07.990607 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.990575 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd2sq\" (UniqueName: \"kubernetes.io/projected/8df8584c-c2db-416a-a790-6a51db7aea0e-kube-api-access-hd2sq\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:07.991599 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.991575 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvv8c\" (UniqueName: \"kubernetes.io/projected/3e4c6b0f-837d-4eca-b8f4-c922e1de910d-kube-api-access-jvv8c\") pod \"volume-data-source-validator-5c4db8bdf9-c8pst\" (UID: \"3e4c6b0f-837d-4eca-b8f4-c922e1de910d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-c8pst" Sep 17 19:07:07.999239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:07.999215 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" Sep 17 19:07:08.079173 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.079050 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-default-certificate\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:08.079173 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.079174 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpj56\" (UniqueName: \"kubernetes.io/projected/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-kube-api-access-lpj56\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:08.079402 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.079255 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:08.079402 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.079286 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:08.079402 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.079316 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-stats-auth\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:08.079537 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:08.079427 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 19:07:08.079537 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:08.079469 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:08.579448876 +0000 UTC m=+36.841322279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : configmap references non-existent config key: service-ca.crt Sep 17 19:07:08.079537 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:08.079491 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:08.579481139 +0000 UTC m=+36.841354523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : secret "router-metrics-certs-default" not found Sep 17 19:07:08.082468 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.082441 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-stats-auth\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:08.082589 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.082471 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-default-certificate\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:08.090144 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.090087 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpj56\" (UniqueName: \"kubernetes.io/projected/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-kube-api-access-lpj56\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:08.097245 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.097223 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-c8pst" Sep 17 19:07:08.382804 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.382713 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:08.382957 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:08.382854 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 19:07:08.382957 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:08.382928 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls podName:1a196995-e126-4724-b2b5-2b108862b410 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:09.382906932 +0000 UTC m=+37.644780329 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7nc5s" (UID: "1a196995-e126-4724-b2b5-2b108862b410") : secret "samples-operator-tls" not found Sep 17 19:07:08.414391 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.414353 2563 generic.go:356] "Generic (PLEG): container finished" podID="8b46af12-297a-4da0-b3f3-730424604854" containerID="c7f4aa4531d82c0d1c5cf22cc2180010e6140f38f36456d66f06148add7744db" exitCode=0 Sep 17 19:07:08.414569 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.414426 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" event={"ID":"8b46af12-297a-4da0-b3f3-730424604854","Type":"ContainerDied","Data":"c7f4aa4531d82c0d1c5cf22cc2180010e6140f38f36456d66f06148add7744db"} Sep 17 19:07:08.484210 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.484170 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:08.484482 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:08.484463 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:08.484586 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:08.484564 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls podName:8df8584c-c2db-416a-a790-6a51db7aea0e nodeName:}" failed. No retries permitted until 2026-09-17 19:07:09.484542506 +0000 UTC m=+37.746415910 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-jb6mm" (UID: "8df8584c-c2db-416a-a790-6a51db7aea0e") : secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:08.584860 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.584820 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:08.585028 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.584879 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:08.585028 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:08.584965 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:09.584943748 +0000 UTC m=+37.846817145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : configmap references non-existent config key: service-ca.crt Sep 17 19:07:08.585183 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:08.585038 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 19:07:08.585183 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:08.585128 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:09.585087733 +0000 UTC m=+37.846961131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : secret "router-metrics-certs-default" not found Sep 17 19:07:08.630352 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.630322 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-5c69q"] Sep 17 19:07:08.645384 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.644914 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.649116 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.649073 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 17 19:07:08.649335 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.649318 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gg24s\"" Sep 17 19:07:08.650626 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.650420 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 17 19:07:08.653345 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.653310 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 17 19:07:08.653474 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.653310 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 17 19:07:08.656147 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.656117 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-5c69q"] Sep 17 19:07:08.661692 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.661663 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 17 19:07:08.787229 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.787005 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-config\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.787874 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.787249 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-serving-cert\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.787874 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.787277 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-trusted-ca\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.787874 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.787352 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsfcz\" (UniqueName: \"kubernetes.io/projected/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-kube-api-access-tsfcz\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.888413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.888379 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-serving-cert\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.888413 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.888410 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-trusted-ca\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.888567 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.888454 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsfcz\" (UniqueName: \"kubernetes.io/projected/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-kube-api-access-tsfcz\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.888567 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.888550 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-config\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.889225 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.889206 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-config\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.889329 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.889314 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-trusted-ca\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.891538 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.891521 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-serving-cert\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.899220 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.899131 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsfcz\" (UniqueName: \"kubernetes.io/projected/e0a5aab2-9dee-4f53-b571-4c0be55c77e0-kube-api-access-tsfcz\") pod \"console-operator-fd4bb46c-5c69q\" (UID: \"e0a5aab2-9dee-4f53-b571-4c0be55c77e0\") " pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:08.934812 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.934692 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-c8pst"] Sep 17 19:07:08.939143 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:08.939089 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e4c6b0f_837d_4eca_b8f4_c922e1de910d.slice/crio-c887282910d68f70265b245ff85ad4040d97f85383d764b666e3a41145043fb8 WatchSource:0}: Error finding container c887282910d68f70265b245ff85ad4040d97f85383d764b666e3a41145043fb8: Status 404 returned error can't find the container with id c887282910d68f70265b245ff85ad4040d97f85383d764b666e3a41145043fb8 Sep 17 19:07:08.945115 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.945074 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-mml6g"] Sep 17 19:07:08.951203 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:08.951180 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0280dd0f_999f_44b2_b052_4652b01c394c.slice/crio-ffd1089c135a15110967ff9a8af36440c47a7e6751b34738c0c5e7013cad4cbf WatchSource:0}: Error finding container ffd1089c135a15110967ff9a8af36440c47a7e6751b34738c0c5e7013cad4cbf: Status 404 returned error can't find the container with id ffd1089c135a15110967ff9a8af36440c47a7e6751b34738c0c5e7013cad4cbf Sep 17 19:07:08.956748 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:08.956717 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:09.252440 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.252409 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-5c69q"] Sep 17 19:07:09.256065 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:09.256038 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a5aab2_9dee_4f53_b571_4c0be55c77e0.slice/crio-4a76ccf7d6cd2804b33a3e829ddb53fce3b03a5bc0a01599917c434d64f02bd5 WatchSource:0}: Error finding container 4a76ccf7d6cd2804b33a3e829ddb53fce3b03a5bc0a01599917c434d64f02bd5: Status 404 returned error can't find the container with id 4a76ccf7d6cd2804b33a3e829ddb53fce3b03a5bc0a01599917c434d64f02bd5 Sep 17 19:07:09.293155 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.293111 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:09.293339 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.293189 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:09.293339 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.293263 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 19:07:09.293339 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.293284 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758b8c48b6-7p8j9: secret "image-registry-tls" not found Sep 17 19:07:09.293339 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.293342 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls podName:2b58566f-f608-4e4d-9e2f-4bd88bb65202 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:13.293327874 +0000 UTC m=+41.555201282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls") pod "image-registry-758b8c48b6-7p8j9" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202") : secret "image-registry-tls" not found Sep 17 19:07:09.293526 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.293356 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:09.293526 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.293430 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls podName:92e91b8e-d907-451f-8653-29bd65f28caa nodeName:}" failed. No retries permitted until 2026-09-17 19:07:13.293413155 +0000 UTC m=+41.555286562 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls") pod "dns-default-fbrxt" (UID: "92e91b8e-d907-451f-8653-29bd65f28caa") : secret "dns-default-metrics-tls" not found Sep 17 19:07:09.393883 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.393852 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:09.394068 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.394036 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 19:07:09.394144 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.394126 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls podName:1a196995-e126-4724-b2b5-2b108862b410 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:11.394088748 +0000 UTC m=+39.655962133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7nc5s" (UID: "1a196995-e126-4724-b2b5-2b108862b410") : secret "samples-operator-tls" not found Sep 17 19:07:09.420326 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.420262 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qlb49" event={"ID":"826c88c0-6862-4bc7-ad50-51c7365ded8b","Type":"ContainerStarted","Data":"42292cfa63ddbf0b079f2b542daaa20ef0e6a1bd947f6d4360cc2b96ffbf7249"} Sep 17 19:07:09.421750 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.421718 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-c8pst" event={"ID":"3e4c6b0f-837d-4eca-b8f4-c922e1de910d","Type":"ContainerStarted","Data":"c887282910d68f70265b245ff85ad4040d97f85383d764b666e3a41145043fb8"} Sep 17 19:07:09.423328 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.423302 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" event={"ID":"0280dd0f-999f-44b2-b052-4652b01c394c","Type":"ContainerStarted","Data":"ffd1089c135a15110967ff9a8af36440c47a7e6751b34738c0c5e7013cad4cbf"} Sep 17 19:07:09.424883 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.424859 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" event={"ID":"e0a5aab2-9dee-4f53-b571-4c0be55c77e0","Type":"ContainerStarted","Data":"4a76ccf7d6cd2804b33a3e829ddb53fce3b03a5bc0a01599917c434d64f02bd5"} Sep 17 19:07:09.429272 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.429249 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" event={"ID":"8b46af12-297a-4da0-b3f3-730424604854","Type":"ContainerStarted","Data":"eddcb5d8bd2cf5abd3edd0012b1c3a83884bddaf07c0aa0d183ab94a1d681eb5"} Sep 17 19:07:09.446550 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.446486 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qlb49" podStartSLOduration=2.028891299 podStartE2EDuration="4.446464297s" totalImagesPullingTime="2.417572998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:09.445267827 +0000 UTC m=+37.707141250" watchObservedRunningTime="2026-09-17 19:07:09.446464297 +0000 UTC m=+37.708337708" Sep 17 19:07:09.489215 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.489153 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-f9wcr" podStartSLOduration=8.903290299 podStartE2EDuration="37.489129343s" totalImagesPullingTime="28.585839044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:09.484422637 +0000 UTC m=+37.746296069" watchObservedRunningTime="2026-09-17 19:07:09.489129343 +0000 UTC m=+37.751002751" Sep 17 19:07:09.494886 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.494836 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:09.495034 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.495013 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:09.495538 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.495118 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls podName:8df8584c-c2db-416a-a790-6a51db7aea0e nodeName:}" failed. No retries permitted until 2026-09-17 19:07:11.495076823 +0000 UTC m=+39.756950226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-jb6mm" (UID: "8df8584c-c2db-416a-a790-6a51db7aea0e") : secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:09.596593 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.596493 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:09.596593 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:09.596558 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:09.596884 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.596841 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 19:07:09.596953 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.596879 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:11.596855118 +0000 UTC m=+39.858728527 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : configmap references non-existent config key: service-ca.crt Sep 17 19:07:09.596953 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:09.596929 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:11.596911848 +0000 UTC m=+39.858785250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : secret "router-metrics-certs-default" not found Sep 17 19:07:11.413380 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:11.413292 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:11.413779 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:11.413458 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 19:07:11.413779 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:11.413530 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls podName:1a196995-e126-4724-b2b5-2b108862b410 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:15.413514385 +0000 UTC m=+43.675387783 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7nc5s" (UID: "1a196995-e126-4724-b2b5-2b108862b410") : secret "samples-operator-tls" not found Sep 17 19:07:11.514292 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:11.514252 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:11.514494 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:11.514446 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:11.514564 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:11.514531 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls podName:8df8584c-c2db-416a-a790-6a51db7aea0e nodeName:}" failed. No retries permitted until 2026-09-17 19:07:15.514508459 +0000 UTC m=+43.776381866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-jb6mm" (UID: "8df8584c-c2db-416a-a790-6a51db7aea0e") : secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:11.615398 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:11.615357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:11.615398 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:11.615403 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:11.615619 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:11.615543 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:15.615523399 +0000 UTC m=+43.877396804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : configmap references non-existent config key: service-ca.crt Sep 17 19:07:11.615619 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:11.615562 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 19:07:11.615619 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:11.615598 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:15.615586503 +0000 UTC m=+43.877459904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : secret "router-metrics-certs-default" not found Sep 17 19:07:12.439736 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:12.439637 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-c8pst" event={"ID":"3e4c6b0f-837d-4eca-b8f4-c922e1de910d","Type":"ContainerStarted","Data":"7ef03fb93ffa2dc3a3dd73f888160c3442487408b040b08046eaaf46cac7b304"} Sep 17 19:07:12.441301 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:12.441270 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" event={"ID":"0280dd0f-999f-44b2-b052-4652b01c394c","Type":"ContainerStarted","Data":"9e695da033557cc6c8a16351b5ffe5cb31811ec0fe88bbf3e3b485b2f0f8a592"} Sep 17 19:07:12.442878 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:12.442860 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/0.log" Sep 17 19:07:12.442978 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:12.442891 2563 generic.go:356] "Generic (PLEG): container finished" podID="e0a5aab2-9dee-4f53-b571-4c0be55c77e0" containerID="bb2500d3b88a494ceda485f47fc33934595ce642b2263aba71acf57d512fbf05" exitCode=255 Sep 17 19:07:12.442978 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:12.442947 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" event={"ID":"e0a5aab2-9dee-4f53-b571-4c0be55c77e0","Type":"ContainerDied","Data":"bb2500d3b88a494ceda485f47fc33934595ce642b2263aba71acf57d512fbf05"} Sep 17 19:07:12.443191 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:12.443175 2563 scope.go:117] "RemoveContainer" containerID="bb2500d3b88a494ceda485f47fc33934595ce642b2263aba71acf57d512fbf05" Sep 17 19:07:12.459701 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:12.457367 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-c8pst" podStartSLOduration=2.3411227119999998 podStartE2EDuration="5.457348176s" totalImagesPullingTime="3.116225464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:12.456534889 +0000 UTC m=+40.718408319" watchObservedRunningTime="2026-09-17 19:07:12.457348176 +0000 UTC m=+40.719221583" Sep 17 19:07:13.328731 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.328691 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:13.328893 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.328769 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:13.328893 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:13.328867 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:13.328893 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:13.328875 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 19:07:13.328985 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:13.328900 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758b8c48b6-7p8j9: secret "image-registry-tls" not found Sep 17 19:07:13.328985 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:13.328929 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls podName:92e91b8e-d907-451f-8653-29bd65f28caa nodeName:}" failed. No retries permitted until 2026-09-17 19:07:21.32891598 +0000 UTC m=+49.590789365 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls") pod "dns-default-fbrxt" (UID: "92e91b8e-d907-451f-8653-29bd65f28caa") : secret "dns-default-metrics-tls" not found Sep 17 19:07:13.329054 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:13.328990 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls podName:2b58566f-f608-4e4d-9e2f-4bd88bb65202 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:21.328971167 +0000 UTC m=+49.590844567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls") pod "image-registry-758b8c48b6-7p8j9" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202") : secret "image-registry-tls" not found Sep 17 19:07:13.447707 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.447631 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:07:13.448148 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.448121 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/0.log" Sep 17 19:07:13.448197 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.448152 2563 generic.go:356] "Generic (PLEG): container finished" podID="e0a5aab2-9dee-4f53-b571-4c0be55c77e0" containerID="6dbd396f89497db2928f32a8c028d2d6cb5692e64dbef60982d6f44182b8a72c" exitCode=255 Sep 17 19:07:13.448282 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.448261 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" event={"ID":"e0a5aab2-9dee-4f53-b571-4c0be55c77e0","Type":"ContainerDied","Data":"6dbd396f89497db2928f32a8c028d2d6cb5692e64dbef60982d6f44182b8a72c"} Sep 17 19:07:13.448366 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.448353 2563 scope.go:117] "RemoveContainer" containerID="bb2500d3b88a494ceda485f47fc33934595ce642b2263aba71acf57d512fbf05" Sep 17 19:07:13.448612 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.448589 2563 scope.go:117] "RemoveContainer" containerID="6dbd396f89497db2928f32a8c028d2d6cb5692e64dbef60982d6f44182b8a72c" Sep 17 19:07:13.448794 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:13.448775 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-5c69q_openshift-console-operator(e0a5aab2-9dee-4f53-b571-4c0be55c77e0)\"" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" podUID="e0a5aab2-9dee-4f53-b571-4c0be55c77e0" Sep 17 19:07:13.472500 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.472447 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" podStartSLOduration=3.341135834 podStartE2EDuration="6.47242941s" totalImagesPullingTime="3.131293576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:12.523169537 +0000 UTC m=+40.785042959" watchObservedRunningTime="2026-09-17 19:07:13.47242941 +0000 UTC m=+41.734302817" Sep 17 19:07:13.842959 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.842928 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6lp9m"] Sep 17 19:07:13.875697 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.875662 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6lp9m"] Sep 17 19:07:13.875853 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.875800 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:13.879863 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.879824 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 17 19:07:13.880023 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.879989 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r9nrg\"" Sep 17 19:07:13.880123 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:13.880057 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 17 19:07:14.035630 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.035590 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfeffc39-8808-4e44-b91c-7986b85a38a6-data-volume\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.035630 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.035633 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.035837 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.035690 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng2s6\" (UniqueName: \"kubernetes.io/projected/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-api-access-ng2s6\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.035837 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.035739 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfeffc39-8808-4e44-b91c-7986b85a38a6-crio-socket\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.035837 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.035763 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.136650 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.136564 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfeffc39-8808-4e44-b91c-7986b85a38a6-data-volume\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.136650 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.136600 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.136875 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.136666 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng2s6\" (UniqueName: \"kubernetes.io/projected/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-api-access-ng2s6\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.136875 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.136694 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfeffc39-8808-4e44-b91c-7986b85a38a6-crio-socket\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.136875 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.136713 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.136875 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:14.136772 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 19:07:14.136875 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.136801 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfeffc39-8808-4e44-b91c-7986b85a38a6-crio-socket\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.136875 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:14.136806 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 19:07:14.136875 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:14.136856 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm podName:cfeffc39-8808-4e44-b91c-7986b85a38a6 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:14.636834217 +0000 UTC m=+42.898707626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6lp9m" (UID: "cfeffc39-8808-4e44-b91c-7986b85a38a6") : configmap "kube-rbac-proxy" not found Sep 17 19:07:14.136875 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:14.136876 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls podName:cfeffc39-8808-4e44-b91c-7986b85a38a6 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:14.63686702 +0000 UTC m=+42.898740407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6lp9m" (UID: "cfeffc39-8808-4e44-b91c-7986b85a38a6") : secret "insights-runtime-extractor-tls" not found Sep 17 19:07:14.137189 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.136983 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfeffc39-8808-4e44-b91c-7986b85a38a6-data-volume\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.149593 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.149564 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng2s6\" (UniqueName: \"kubernetes.io/projected/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-api-access-ng2s6\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.452343 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.452267 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:07:14.452670 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.452587 2563 scope.go:117] "RemoveContainer" containerID="6dbd396f89497db2928f32a8c028d2d6cb5692e64dbef60982d6f44182b8a72c" Sep 17 19:07:14.452753 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:14.452734 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-5c69q_openshift-console-operator(e0a5aab2-9dee-4f53-b571-4c0be55c77e0)\"" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" podUID="e0a5aab2-9dee-4f53-b571-4c0be55c77e0" Sep 17 19:07:14.641573 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.641530 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.641745 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:14.641598 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:14.641745 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:14.641661 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 19:07:14.641745 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:14.641729 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm podName:cfeffc39-8808-4e44-b91c-7986b85a38a6 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:15.641711706 +0000 UTC m=+43.903585105 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6lp9m" (UID: "cfeffc39-8808-4e44-b91c-7986b85a38a6") : configmap "kube-rbac-proxy" not found Sep 17 19:07:14.641941 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:14.641793 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 19:07:14.641941 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:14.641875 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls podName:cfeffc39-8808-4e44-b91c-7986b85a38a6 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:15.641857098 +0000 UTC m=+43.903730500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6lp9m" (UID: "cfeffc39-8808-4e44-b91c-7986b85a38a6") : secret "insights-runtime-extractor-tls" not found Sep 17 19:07:15.449134 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:15.449076 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:15.449320 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:15.449223 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 19:07:15.449320 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:15.449291 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls podName:1a196995-e126-4724-b2b5-2b108862b410 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:23.449276078 +0000 UTC m=+51.711149466 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7nc5s" (UID: "1a196995-e126-4724-b2b5-2b108862b410") : secret "samples-operator-tls" not found Sep 17 19:07:15.549688 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:15.549649 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:15.550192 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:15.549809 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:15.550192 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:15.549872 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls podName:8df8584c-c2db-416a-a790-6a51db7aea0e nodeName:}" failed. No retries permitted until 2026-09-17 19:07:23.549857418 +0000 UTC m=+51.811730803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-jb6mm" (UID: "8df8584c-c2db-416a-a790-6a51db7aea0e") : secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:15.651032 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:15.650995 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:15.651237 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:15.651041 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:15.651237 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:15.651062 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:15.651237 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:15.651158 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 19:07:15.651237 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:15.651193 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 19:07:15.651237 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:15.651204 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:23.651188333 +0000 UTC m=+51.913061744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : configmap references non-existent config key: service-ca.crt Sep 17 19:07:15.651237 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:15.651222 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls podName:cfeffc39-8808-4e44-b91c-7986b85a38a6 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:17.651215911 +0000 UTC m=+45.913089296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6lp9m" (UID: "cfeffc39-8808-4e44-b91c-7986b85a38a6") : secret "insights-runtime-extractor-tls" not found Sep 17 19:07:15.651461 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:15.651263 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:23.651248152 +0000 UTC m=+51.913121554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : secret "router-metrics-certs-default" not found Sep 17 19:07:15.651461 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:15.651306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:15.651461 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:15.651378 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 19:07:15.651461 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:15.651419 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm podName:cfeffc39-8808-4e44-b91c-7986b85a38a6 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:17.651408801 +0000 UTC m=+45.913282202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6lp9m" (UID: "cfeffc39-8808-4e44-b91c-7986b85a38a6") : configmap "kube-rbac-proxy" not found Sep 17 19:07:15.760010 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:15.759982 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l6h2d_ac28db00-3290-4731-ab0c-9963ef1d6560/dns-node-resolver/0.log" Sep 17 19:07:16.959074 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:16.959047 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q89g2_76086d58-ef7d-4a2c-9614-c15931bac3df/node-ca/0.log" Sep 17 19:07:17.559521 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:17.559493 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qlb49_826c88c0-6862-4bc7-ad50-51c7365ded8b/serve-healthcheck-canary/0.log" Sep 17 19:07:17.668634 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:17.668595 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:17.668782 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:17.668729 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:17.668782 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:17.668754 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 19:07:17.668888 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:17.668799 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 19:07:17.668888 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:17.668833 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls podName:cfeffc39-8808-4e44-b91c-7986b85a38a6 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:21.668818092 +0000 UTC m=+49.930691491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6lp9m" (UID: "cfeffc39-8808-4e44-b91c-7986b85a38a6") : secret "insights-runtime-extractor-tls" not found Sep 17 19:07:17.668888 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:17.668848 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm podName:cfeffc39-8808-4e44-b91c-7986b85a38a6 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:21.668841895 +0000 UTC m=+49.930715279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6lp9m" (UID: "cfeffc39-8808-4e44-b91c-7986b85a38a6") : configmap "kube-rbac-proxy" not found Sep 17 19:07:18.957508 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:18.957479 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:18.957846 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:18.957530 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:18.957880 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:18.957864 2563 scope.go:117] "RemoveContainer" containerID="6dbd396f89497db2928f32a8c028d2d6cb5692e64dbef60982d6f44182b8a72c" Sep 17 19:07:18.958056 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:18.958037 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-5c69q_openshift-console-operator(e0a5aab2-9dee-4f53-b571-4c0be55c77e0)\"" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" podUID="e0a5aab2-9dee-4f53-b571-4c0be55c77e0" Sep 17 19:07:21.399136 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:21.399075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:21.399604 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:21.399207 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:21.399604 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:21.399223 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:21.399604 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:21.399288 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 19:07:21.399604 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:21.399299 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758b8c48b6-7p8j9: secret "image-registry-tls" not found Sep 17 19:07:21.399604 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:21.399311 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls podName:92e91b8e-d907-451f-8653-29bd65f28caa nodeName:}" failed. No retries permitted until 2026-09-17 19:07:37.399294751 +0000 UTC m=+65.661168157 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls") pod "dns-default-fbrxt" (UID: "92e91b8e-d907-451f-8653-29bd65f28caa") : secret "dns-default-metrics-tls" not found Sep 17 19:07:21.399604 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:21.399336 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls podName:2b58566f-f608-4e4d-9e2f-4bd88bb65202 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:37.399325413 +0000 UTC m=+65.661198799 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls") pod "image-registry-758b8c48b6-7p8j9" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202") : secret "image-registry-tls" not found Sep 17 19:07:21.701689 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:21.701600 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:21.701689 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:21.701667 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:21.701873 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:21.701733 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 19:07:21.701873 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:21.701783 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 19:07:21.701873 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:21.701801 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm podName:cfeffc39-8808-4e44-b91c-7986b85a38a6 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:29.70178587 +0000 UTC m=+57.963659275 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6lp9m" (UID: "cfeffc39-8808-4e44-b91c-7986b85a38a6") : configmap "kube-rbac-proxy" not found Sep 17 19:07:21.701873 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:21.701835 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls podName:cfeffc39-8808-4e44-b91c-7986b85a38a6 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:29.701823955 +0000 UTC m=+57.963697374 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6lp9m" (UID: "cfeffc39-8808-4e44-b91c-7986b85a38a6") : secret "insights-runtime-extractor-tls" not found Sep 17 19:07:23.517768 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:23.517723 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:23.518208 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:23.517896 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 19:07:23.518208 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:23.517993 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls podName:1a196995-e126-4724-b2b5-2b108862b410 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:39.517965177 +0000 UTC m=+67.779838581 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-7nc5s" (UID: "1a196995-e126-4724-b2b5-2b108862b410") : secret "samples-operator-tls" not found Sep 17 19:07:23.618414 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:23.618376 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:23.618576 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:23.618533 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:23.618617 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:23.618605 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls podName:8df8584c-c2db-416a-a790-6a51db7aea0e nodeName:}" failed. No retries permitted until 2026-09-17 19:07:39.618587817 +0000 UTC m=+67.880461222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-jb6mm" (UID: "8df8584c-c2db-416a-a790-6a51db7aea0e") : secret "cluster-monitoring-operator-tls" not found Sep 17 19:07:23.719566 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:23.719528 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:23.719566 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:23.719570 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:23.719832 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:23.719675 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 19:07:23.719832 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:23.719720 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:39.719697841 +0000 UTC m=+67.981571245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : configmap references non-existent config key: service-ca.crt Sep 17 19:07:23.719832 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:23.719748 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs podName:7213bf71-54de-4fc3-8fa0-89b2e1d45f51 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:39.719736933 +0000 UTC m=+67.981610319 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs") pod "router-default-544cd45fb-8wb98" (UID: "7213bf71-54de-4fc3-8fa0-89b2e1d45f51") : secret "router-metrics-certs-default" not found Sep 17 19:07:29.768067 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:29.768023 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:29.768658 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:29.768089 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:29.768729 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:29.768695 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfeffc39-8808-4e44-b91c-7986b85a38a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:29.770569 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:29.770547 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfeffc39-8808-4e44-b91c-7986b85a38a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lp9m\" (UID: \"cfeffc39-8808-4e44-b91c-7986b85a38a6\") " pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:29.784587 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:29.784554 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6lp9m" Sep 17 19:07:30.090616 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:30.090540 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6lp9m"] Sep 17 19:07:30.093849 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:30.093812 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfeffc39_8808_4e44_b91c_7986b85a38a6.slice/crio-23231532f9a6b5efa17e7be6810f855c01faa37fb8fe649a255142a52781ab44 WatchSource:0}: Error finding container 23231532f9a6b5efa17e7be6810f855c01faa37fb8fe649a255142a52781ab44: Status 404 returned error can't find the container with id 23231532f9a6b5efa17e7be6810f855c01faa37fb8fe649a255142a52781ab44 Sep 17 19:07:30.394027 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:30.393932 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m5854" Sep 17 19:07:30.496446 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:30.496412 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6lp9m" event={"ID":"cfeffc39-8808-4e44-b91c-7986b85a38a6","Type":"ContainerStarted","Data":"3faac8bf4bdc7478c8ad13658f4e485813736c49e5774a3875a293cce4a721be"} Sep 17 19:07:30.496609 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:30.496495 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6lp9m" event={"ID":"cfeffc39-8808-4e44-b91c-7986b85a38a6","Type":"ContainerStarted","Data":"23231532f9a6b5efa17e7be6810f855c01faa37fb8fe649a255142a52781ab44"} Sep 17 19:07:31.501349 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:31.501314 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6lp9m" event={"ID":"cfeffc39-8808-4e44-b91c-7986b85a38a6","Type":"ContainerStarted","Data":"e956a0a986ce819bd62cb255bc3cc50e6c5a2c0fb0d4a87bbc11157c5099cf2d"} Sep 17 19:07:32.231174 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:32.231141 2563 scope.go:117] "RemoveContainer" containerID="6dbd396f89497db2928f32a8c028d2d6cb5692e64dbef60982d6f44182b8a72c" Sep 17 19:07:32.509197 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:32.509069 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:07:32.509526 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:32.509209 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" event={"ID":"e0a5aab2-9dee-4f53-b571-4c0be55c77e0","Type":"ContainerStarted","Data":"9cd584a98bf3d4dcde009a52898855032a223fccbcf26cd3353a7bddd4518080"} Sep 17 19:07:32.509591 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:32.509544 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:32.510781 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:32.510758 2563 patch_prober.go:28] interesting pod/console-operator-fd4bb46c-5c69q container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused" start-of-body= Sep 17 19:07:32.510865 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:32.510798 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" podUID="e0a5aab2-9dee-4f53-b571-4c0be55c77e0" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused" Sep 17 19:07:32.531210 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:32.531126 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" podStartSLOduration=21.699760852 podStartE2EDuration="24.531084245s" totalImagesPullingTime="2.831323393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:32.529940284 +0000 UTC m=+60.791813707" watchObservedRunningTime="2026-09-17 19:07:32.531084245 +0000 UTC m=+60.792957653" Sep 17 19:07:33.517240 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:33.517197 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6lp9m" event={"ID":"cfeffc39-8808-4e44-b91c-7986b85a38a6","Type":"ContainerStarted","Data":"59fddedd6dff81c947c25972873ce5de9abe8bc3c5129cfb4f69d49542f372b1"} Sep 17 19:07:33.521265 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:33.521241 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-5c69q" Sep 17 19:07:33.543909 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:33.543854 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6lp9m" podStartSLOduration=18.410070401 podStartE2EDuration="20.543837725s" totalImagesPullingTime="2.133767324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:07:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:33.543708365 +0000 UTC m=+61.805581771" watchObservedRunningTime="2026-09-17 19:07:33.543837725 +0000 UTC m=+61.805711149" Sep 17 19:07:37.435387 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.435338 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:37.435860 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.435418 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:37.437936 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.437904 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e91b8e-d907-451f-8653-29bd65f28caa-metrics-tls\") pod \"dns-default-fbrxt\" (UID: \"92e91b8e-d907-451f-8653-29bd65f28caa\") " pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:37.438060 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.437957 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls\") pod \"image-registry-758b8c48b6-7p8j9\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:37.620019 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.619988 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-msk9c\"" Sep 17 19:07:37.628242 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.628219 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:37.639991 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.639963 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dpmzt\"" Sep 17 19:07:37.648135 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.648089 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:37.939858 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.939817 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-758b8c48b6-7p8j9"] Sep 17 19:07:37.941848 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.941820 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:07:37.942818 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:37.942795 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b58566f_f608_4e4d_9e2f_4bd88bb65202.slice/crio-84ee2faaf1c470763d0bb839dcfae16f3adaa15f8f2adb47d523b10be2af1135 WatchSource:0}: Error finding container 84ee2faaf1c470763d0bb839dcfae16f3adaa15f8f2adb47d523b10be2af1135: Status 404 returned error can't find the container with id 84ee2faaf1c470763d0bb839dcfae16f3adaa15f8f2adb47d523b10be2af1135 Sep 17 19:07:37.944526 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.944508 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 19:07:37.955244 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.955217 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7d66c218-c6ed-4ca6-a49a-47bdc94fef2d-original-pull-secret\") pod \"global-pull-secret-syncer-xvjpg\" (UID: \"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d\") " pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:07:37.962277 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:37.962257 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fbrxt"] Sep 17 19:07:37.965718 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:37.965690 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e91b8e_d907_451f_8653_29bd65f28caa.slice/crio-e4287894bd3f6eb86678ef78cf8f4648dd9f3ea21688b8ac423781fa0a43f845 WatchSource:0}: Error finding container e4287894bd3f6eb86678ef78cf8f4648dd9f3ea21688b8ac423781fa0a43f845: Status 404 returned error can't find the container with id e4287894bd3f6eb86678ef78cf8f4648dd9f3ea21688b8ac423781fa0a43f845 Sep 17 19:07:38.043160 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.043130 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:07:38.045467 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.045439 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 19:07:38.053795 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.053770 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvjpg" Sep 17 19:07:38.055885 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.055869 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431387c9-5a3c-44d5-bad2-94a08bc73700-metrics-certs\") pod \"network-metrics-daemon-6mspz\" (UID: \"431387c9-5a3c-44d5-bad2-94a08bc73700\") " pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:07:38.059959 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.059940 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-47zq2\"" Sep 17 19:07:38.068222 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.068194 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6mspz" Sep 17 19:07:38.143584 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.143555 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hx9f\" (UniqueName: \"kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f\") pod \"network-check-target-z7q6d\" (UID: \"357e149f-654a-4188-9229-1a440c1c30a9\") " pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:07:38.146427 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.146238 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 19:07:38.156381 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.156362 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 19:07:38.167795 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.167764 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hx9f\" (UniqueName: \"kubernetes.io/projected/357e149f-654a-4188-9229-1a440c1c30a9-kube-api-access-2hx9f\") pod \"network-check-target-z7q6d\" (UID: \"357e149f-654a-4188-9229-1a440c1c30a9\") " pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:07:38.338772 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.338691 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xvjpg"] Sep 17 19:07:38.342073 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:38.342047 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d66c218_c6ed_4ca6_a49a_47bdc94fef2d.slice/crio-1afc70896c64def2f962b509c43ff80e0702d11014895988890b132630f6e120 WatchSource:0}: Error finding container 1afc70896c64def2f962b509c43ff80e0702d11014895988890b132630f6e120: Status 404 returned error can't find the container with id 1afc70896c64def2f962b509c43ff80e0702d11014895988890b132630f6e120 Sep 17 19:07:38.350751 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.350728 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b9ftw\"" Sep 17 19:07:38.358083 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.358056 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:07:38.359471 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.359415 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6mspz"] Sep 17 19:07:38.363128 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:38.363087 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod431387c9_5a3c_44d5_bad2_94a08bc73700.slice/crio-e52762bd1277ab279279a49ebc6f17df035c8eac2ee3e8c0e154721ca407d698 WatchSource:0}: Error finding container e52762bd1277ab279279a49ebc6f17df035c8eac2ee3e8c0e154721ca407d698: Status 404 returned error can't find the container with id e52762bd1277ab279279a49ebc6f17df035c8eac2ee3e8c0e154721ca407d698 Sep 17 19:07:38.537853 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.537779 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fbrxt" event={"ID":"92e91b8e-d907-451f-8653-29bd65f28caa","Type":"ContainerStarted","Data":"e4287894bd3f6eb86678ef78cf8f4648dd9f3ea21688b8ac423781fa0a43f845"} Sep 17 19:07:38.539717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.539689 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xvjpg" event={"ID":"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d","Type":"ContainerStarted","Data":"1afc70896c64def2f962b509c43ff80e0702d11014895988890b132630f6e120"} Sep 17 19:07:38.541234 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.541206 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6mspz" event={"ID":"431387c9-5a3c-44d5-bad2-94a08bc73700","Type":"ContainerStarted","Data":"e52762bd1277ab279279a49ebc6f17df035c8eac2ee3e8c0e154721ca407d698"} Sep 17 19:07:38.543146 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.543123 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" event={"ID":"2b58566f-f608-4e4d-9e2f-4bd88bb65202","Type":"ContainerStarted","Data":"c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e"} Sep 17 19:07:38.543229 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.543160 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" event={"ID":"2b58566f-f608-4e4d-9e2f-4bd88bb65202","Type":"ContainerStarted","Data":"84ee2faaf1c470763d0bb839dcfae16f3adaa15f8f2adb47d523b10be2af1135"} Sep 17 19:07:38.543293 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.543261 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:38.567217 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.567157 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" podStartSLOduration=66.567136051 podStartE2EDuration="1m6.567136051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:38.565267255 +0000 UTC m=+66.827140688" watchObservedRunningTime="2026-09-17 19:07:38.567136051 +0000 UTC m=+66.829009460" Sep 17 19:07:38.648018 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:38.647931 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7q6d"] Sep 17 19:07:38.652209 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:38.652178 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod357e149f_654a_4188_9229_1a440c1c30a9.slice/crio-5eb23c45a07a879d47ef120ab43e6fa75eea15eedcecc3d0e118b511e5bbd441 WatchSource:0}: Error finding container 5eb23c45a07a879d47ef120ab43e6fa75eea15eedcecc3d0e118b511e5bbd441: Status 404 returned error can't find the container with id 5eb23c45a07a879d47ef120ab43e6fa75eea15eedcecc3d0e118b511e5bbd441 Sep 17 19:07:39.550249 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.550164 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7q6d" event={"ID":"357e149f-654a-4188-9229-1a440c1c30a9","Type":"ContainerStarted","Data":"5eb23c45a07a879d47ef120ab43e6fa75eea15eedcecc3d0e118b511e5bbd441"} Sep 17 19:07:39.555788 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.555754 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:39.560608 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.560580 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1a196995-e126-4724-b2b5-2b108862b410-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-7nc5s\" (UID: \"1a196995-e126-4724-b2b5-2b108862b410\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:39.656738 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.656699 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:39.660086 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.660051 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8df8584c-c2db-416a-a790-6a51db7aea0e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-jb6mm\" (UID: \"8df8584c-c2db-416a-a790-6a51db7aea0e\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:39.757976 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.757432 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:39.757976 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.757471 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:39.758663 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.758629 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-service-ca-bundle\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:39.767857 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.767397 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7213bf71-54de-4fc3-8fa0-89b2e1d45f51-metrics-certs\") pod \"router-default-544cd45fb-8wb98\" (UID: \"7213bf71-54de-4fc3-8fa0-89b2e1d45f51\") " pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:39.809194 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.809115 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wrp9n\"" Sep 17 19:07:39.817388 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.817354 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" Sep 17 19:07:39.892803 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.892758 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-n9sxx\"" Sep 17 19:07:39.901073 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:39.901046 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" Sep 17 19:07:40.014072 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:40.014040 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-65f9b\"" Sep 17 19:07:40.022545 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:40.022513 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:41.003894 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:41.003860 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s"] Sep 17 19:07:41.007604 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:41.005480 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-544cd45fb-8wb98"] Sep 17 19:07:41.007604 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:41.006755 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm"] Sep 17 19:07:41.560335 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:41.560297 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6mspz" event={"ID":"431387c9-5a3c-44d5-bad2-94a08bc73700","Type":"ContainerStarted","Data":"fcd85b6489409a779a1f3746e9103f226a07bf75607c672a03b7e8c31d749d05"} Sep 17 19:07:41.560533 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:41.560363 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6mspz" event={"ID":"431387c9-5a3c-44d5-bad2-94a08bc73700","Type":"ContainerStarted","Data":"3d10f10481888507c088a5b23fcc8972abeedac159b03d8980bccaf6bf603dd0"} Sep 17 19:07:41.562459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:41.562425 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fbrxt" event={"ID":"92e91b8e-d907-451f-8653-29bd65f28caa","Type":"ContainerStarted","Data":"9fc442014222ea673c93dc84a57fdba9dae6d41ef9f20d3cea941b1f5807ead7"} Sep 17 19:07:41.562580 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:41.562487 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fbrxt" event={"ID":"92e91b8e-d907-451f-8653-29bd65f28caa","Type":"ContainerStarted","Data":"3b21f0c2f76b70f4deaa8d737aee09908f3e10a141dadc1fd0e523efdf05fa08"} Sep 17 19:07:41.562580 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:41.562512 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:41.581147 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:41.581071 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6mspz" podStartSLOduration=67.296993504 podStartE2EDuration="1m9.581051263s" totalImagesPullingTime="2.284057759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:41.580486228 +0000 UTC m=+69.842359653" watchObservedRunningTime="2026-09-17 19:07:41.581051263 +0000 UTC m=+69.842924671" Sep 17 19:07:41.600306 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:41.600245 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fbrxt" podStartSLOduration=33.919152448 podStartE2EDuration="36.60022313s" totalImagesPullingTime="2.681070682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:41.598791549 +0000 UTC m=+69.860665011" watchObservedRunningTime="2026-09-17 19:07:41.60022313 +0000 UTC m=+69.862096542" Sep 17 19:07:42.056946 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:42.056912 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7213bf71_54de_4fc3_8fa0_89b2e1d45f51.slice/crio-8802c2c9ab20f0d456f111532b550337e242b5e6246204682ece7bffb12b0edd WatchSource:0}: Error finding container 8802c2c9ab20f0d456f111532b550337e242b5e6246204682ece7bffb12b0edd: Status 404 returned error can't find the container with id 8802c2c9ab20f0d456f111532b550337e242b5e6246204682ece7bffb12b0edd Sep 17 19:07:42.058780 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:42.058756 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8df8584c_c2db_416a_a790_6a51db7aea0e.slice/crio-50cb746b3c1ec590eed63588a65e176a6994417820c4b064a6dd224b1e012364 WatchSource:0}: Error finding container 50cb746b3c1ec590eed63588a65e176a6994417820c4b064a6dd224b1e012364: Status 404 returned error can't find the container with id 50cb746b3c1ec590eed63588a65e176a6994417820c4b064a6dd224b1e012364 Sep 17 19:07:42.567522 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:42.567478 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" event={"ID":"1a196995-e126-4724-b2b5-2b108862b410","Type":"ContainerStarted","Data":"9e88988b3e65c5f6437fe884722f5ba60079a150255d8dbadda878fba7ba7298"} Sep 17 19:07:42.569000 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:42.568970 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-544cd45fb-8wb98" event={"ID":"7213bf71-54de-4fc3-8fa0-89b2e1d45f51","Type":"ContainerStarted","Data":"8802c2c9ab20f0d456f111532b550337e242b5e6246204682ece7bffb12b0edd"} Sep 17 19:07:42.570295 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:42.570271 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" event={"ID":"8df8584c-c2db-416a-a790-6a51db7aea0e","Type":"ContainerStarted","Data":"50cb746b3c1ec590eed63588a65e176a6994417820c4b064a6dd224b1e012364"} Sep 17 19:07:43.576308 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:43.575858 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7q6d" event={"ID":"357e149f-654a-4188-9229-1a440c1c30a9","Type":"ContainerStarted","Data":"6e5ba5b11d3c88bcef6a6bd5c33b4443aced76bff94627b8281cf657addadf95"} Sep 17 19:07:43.576308 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:43.575942 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:07:43.579199 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:43.579139 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-544cd45fb-8wb98" event={"ID":"7213bf71-54de-4fc3-8fa0-89b2e1d45f51","Type":"ContainerStarted","Data":"c09324600d587d09151ac5707d5b4346f84c0ea62502a24233a7dafbc42bf49e"} Sep 17 19:07:43.597783 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:43.597275 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z7q6d" podStartSLOduration=66.748903785 podStartE2EDuration="1m11.597254686s" totalImagesPullingTime="4.848350901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:43.596160175 +0000 UTC m=+71.858033607" watchObservedRunningTime="2026-09-17 19:07:43.597254686 +0000 UTC m=+71.859128092" Sep 17 19:07:43.619133 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:43.618884 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-544cd45fb-8wb98" podStartSLOduration=36.618858088 podStartE2EDuration="36.618858088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:43.616645043 +0000 UTC m=+71.878518474" watchObservedRunningTime="2026-09-17 19:07:43.618858088 +0000 UTC m=+71.880731496" Sep 17 19:07:44.022779 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:44.022742 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:44.025829 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:44.025797 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:44.585571 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:44.585527 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xvjpg" event={"ID":"7d66c218-c6ed-4ca6-a49a-47bdc94fef2d","Type":"ContainerStarted","Data":"03eab366fea2ed9b1832c276a719bc6026defe971181afb8bfa40dc159d68ba8"} Sep 17 19:07:44.586059 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:44.585859 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:44.587440 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:44.587416 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-544cd45fb-8wb98" Sep 17 19:07:44.603112 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:44.603054 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xvjpg" podStartSLOduration=67.442596897 podStartE2EDuration="1m12.603034238s" totalImagesPullingTime="5.160437341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:44.601581279 +0000 UTC m=+72.863454721" watchObservedRunningTime="2026-09-17 19:07:44.603034238 +0000 UTC m=+72.864907647" Sep 17 19:07:45.591540 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:45.591441 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" event={"ID":"8df8584c-c2db-416a-a790-6a51db7aea0e","Type":"ContainerStarted","Data":"1e9123ddf9ffc619fceb52997761939ffccf84a2f841a48449a06c78e6d70c69"} Sep 17 19:07:45.593368 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:45.593340 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" event={"ID":"1a196995-e126-4724-b2b5-2b108862b410","Type":"ContainerStarted","Data":"07203c948e91e8c4369df71652f5da9a27439b9b1436f90b197997fe1432df68"} Sep 17 19:07:45.593482 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:45.593389 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" event={"ID":"1a196995-e126-4724-b2b5-2b108862b410","Type":"ContainerStarted","Data":"66209e0ab2329de53a01bddd68a7edf303b207035da2be9ee19da445f38b71ed"} Sep 17 19:07:45.612485 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:45.612439 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-jb6mm" podStartSLOduration=35.463865374 podStartE2EDuration="38.612420797s" totalImagesPullingTime="3.148555423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:45.611917195 +0000 UTC m=+73.873790601" watchObservedRunningTime="2026-09-17 19:07:45.612420797 +0000 UTC m=+73.874294203" Sep 17 19:07:45.632211 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:45.632159 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-7nc5s" podStartSLOduration=35.555217979 podStartE2EDuration="38.632141421s" totalImagesPullingTime="3.076923442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:45.630889979 +0000 UTC m=+73.892763398" watchObservedRunningTime="2026-09-17 19:07:45.632141421 +0000 UTC m=+73.894014829" Sep 17 19:07:50.845814 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:50.845780 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq"] Sep 17 19:07:50.848423 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:50.848407 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:50.851071 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:50.851050 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2d8w9\"" Sep 17 19:07:50.851574 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:50.851558 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 17 19:07:50.851923 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:50.851903 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 17 19:07:50.852019 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:50.852004 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 17 19:07:50.861508 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:50.861486 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq"] Sep 17 19:07:50.952430 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:50.952388 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3a605cb-4839-472d-9890-3fe41ae3dd44-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:50.952593 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:50.952440 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg7tq\" (UniqueName: \"kubernetes.io/projected/c3a605cb-4839-472d-9890-3fe41ae3dd44-kube-api-access-tg7tq\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:50.952593 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:50.952499 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3a605cb-4839-472d-9890-3fe41ae3dd44-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:50.952593 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:50.952546 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3a605cb-4839-472d-9890-3fe41ae3dd44-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:51.053392 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.053359 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3a605cb-4839-472d-9890-3fe41ae3dd44-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:51.053570 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.053400 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg7tq\" (UniqueName: \"kubernetes.io/projected/c3a605cb-4839-472d-9890-3fe41ae3dd44-kube-api-access-tg7tq\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:51.053570 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.053418 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3a605cb-4839-472d-9890-3fe41ae3dd44-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:51.053570 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.053441 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3a605cb-4839-472d-9890-3fe41ae3dd44-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:51.054062 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.054041 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3a605cb-4839-472d-9890-3fe41ae3dd44-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:51.056160 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.056136 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3a605cb-4839-472d-9890-3fe41ae3dd44-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:51.056160 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.056154 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3a605cb-4839-472d-9890-3fe41ae3dd44-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:51.062662 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.062639 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg7tq\" (UniqueName: \"kubernetes.io/projected/c3a605cb-4839-472d-9890-3fe41ae3dd44-kube-api-access-tg7tq\") pod \"prometheus-operator-7bd5d56d6-lpfwq\" (UID: \"c3a605cb-4839-472d-9890-3fe41ae3dd44\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:51.157798 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.157716 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" Sep 17 19:07:51.431235 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.431129 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq"] Sep 17 19:07:51.434512 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:51.434480 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a605cb_4839_472d_9890_3fe41ae3dd44.slice/crio-224487f627b55018968b49077640da73b077d4bb28d4348f6b207518d8bc784d WatchSource:0}: Error finding container 224487f627b55018968b49077640da73b077d4bb28d4348f6b207518d8bc784d: Status 404 returned error can't find the container with id 224487f627b55018968b49077640da73b077d4bb28d4348f6b207518d8bc784d Sep 17 19:07:51.572492 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.572457 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fbrxt" Sep 17 19:07:51.621169 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:51.621131 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" event={"ID":"c3a605cb-4839-472d-9890-3fe41ae3dd44","Type":"ContainerStarted","Data":"224487f627b55018968b49077640da73b077d4bb28d4348f6b207518d8bc784d"} Sep 17 19:07:53.630631 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:53.630490 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" event={"ID":"c3a605cb-4839-472d-9890-3fe41ae3dd44","Type":"ContainerStarted","Data":"68c1c9ce1ccdc60c607062f3a750602e04f3af71a767a5fcc5f8482f8ed54304"} Sep 17 19:07:53.630631 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:53.630571 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" event={"ID":"c3a605cb-4839-472d-9890-3fe41ae3dd44","Type":"ContainerStarted","Data":"73310b7583fdc1751450937dc8599a0cd946691c7904f09da27db99d3befc7ce"} Sep 17 19:07:55.347818 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.347713 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-lpfwq" podStartSLOduration=4.199141124 podStartE2EDuration="5.34768749s" totalImagesPullingTime="1.148546366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:53.651213233 +0000 UTC m=+81.913086641" watchObservedRunningTime="2026-09-17 19:07:55.34768749 +0000 UTC m=+83.609560897" Sep 17 19:07:55.349354 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.349327 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-qfpj5"] Sep 17 19:07:55.354174 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.354157 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.357194 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.357176 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 17 19:07:55.357936 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.357919 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 17 19:07:55.358067 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.358049 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 17 19:07:55.358146 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.358136 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-wvxz8\"" Sep 17 19:07:55.367376 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.367354 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zrzgc"] Sep 17 19:07:55.370477 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.370458 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.374529 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.374506 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 17 19:07:55.376377 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.376356 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-qfpj5"] Sep 17 19:07:55.378251 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.378233 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 17 19:07:55.378359 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.378271 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 17 19:07:55.378462 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.378436 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2zq9r\"" Sep 17 19:07:55.488244 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488201 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-tls\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.488244 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488237 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e16b1bb-d157-4f46-85c4-6ce3e028886d-metrics-client-ca\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.488495 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488259 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-wtmp\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.488495 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488277 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e16b1bb-d157-4f46-85c4-6ce3e028886d-sys\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.488495 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488348 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhjgh\" (UniqueName: \"kubernetes.io/projected/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-api-access-xhjgh\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.488495 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488404 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxqmg\" (UniqueName: \"kubernetes.io/projected/7e16b1bb-d157-4f46-85c4-6ce3e028886d-kube-api-access-cxqmg\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.488495 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488437 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.488495 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488453 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e16b1bb-d157-4f46-85c4-6ce3e028886d-root\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.488495 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488469 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-accelerators-collector-config\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.488795 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488552 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.488795 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5842211a-e5f4-46a5-abdc-765ec50dcd98-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.488795 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488627 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-textfile\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.488795 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488658 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.488795 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488687 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.488795 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.488737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5842211a-e5f4-46a5-abdc-765ec50dcd98-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.589736 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589693 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5842211a-e5f4-46a5-abdc-765ec50dcd98-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.589736 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589739 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-tls\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.589988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589756 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e16b1bb-d157-4f46-85c4-6ce3e028886d-metrics-client-ca\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.589988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589772 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-wtmp\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.589988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589788 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e16b1bb-d157-4f46-85c4-6ce3e028886d-sys\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.589988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589814 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhjgh\" (UniqueName: \"kubernetes.io/projected/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-api-access-xhjgh\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.589988 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:55.589863 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 17 19:07:55.589988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589888 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqmg\" (UniqueName: \"kubernetes.io/projected/7e16b1bb-d157-4f46-85c4-6ce3e028886d-kube-api-access-cxqmg\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.589988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589920 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.589988 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:07:55.589933 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-tls podName:7e16b1bb-d157-4f46-85c4-6ce3e028886d nodeName:}" failed. No retries permitted until 2026-09-17 19:07:56.089916026 +0000 UTC m=+84.351789434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-tls") pod "node-exporter-zrzgc" (UID: "7e16b1bb-d157-4f46-85c4-6ce3e028886d") : secret "node-exporter-tls" not found Sep 17 19:07:55.589988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589934 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-wtmp\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.589988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589957 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e16b1bb-d157-4f46-85c4-6ce3e028886d-root\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.589988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589996 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e16b1bb-d157-4f46-85c4-6ce3e028886d-root\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.590514 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589995 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-accelerators-collector-config\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.590514 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.589868 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e16b1bb-d157-4f46-85c4-6ce3e028886d-sys\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.590610 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.590529 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.590610 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.590582 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5842211a-e5f4-46a5-abdc-765ec50dcd98-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.590697 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.590627 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-textfile\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.590697 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.590667 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.590871 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.590848 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e16b1bb-d157-4f46-85c4-6ce3e028886d-metrics-client-ca\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.590925 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.590853 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-accelerators-collector-config\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.590974 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.590954 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5842211a-e5f4-46a5-abdc-765ec50dcd98-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.591865 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.591763 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-textfile\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.591865 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.591812 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5842211a-e5f4-46a5-abdc-765ec50dcd98-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.592072 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.591940 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.595960 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.592585 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.595960 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.593830 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.595960 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.593990 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.595960 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.594221 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.607533 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.607475 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxqmg\" (UniqueName: \"kubernetes.io/projected/7e16b1bb-d157-4f46-85c4-6ce3e028886d-kube-api-access-cxqmg\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:55.612939 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.612912 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhjgh\" (UniqueName: \"kubernetes.io/projected/5842211a-e5f4-46a5-abdc-765ec50dcd98-kube-api-access-xhjgh\") pod \"kube-state-metrics-84c885658c-qfpj5\" (UID: \"5842211a-e5f4-46a5-abdc-765ec50dcd98\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.663262 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.663226 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" Sep 17 19:07:55.943685 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:55.943660 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-qfpj5"] Sep 17 19:07:55.946015 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:55.945986 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5842211a_e5f4_46a5_abdc_765ec50dcd98.slice/crio-e2c665cc3aa61787d35a53d97f5f7d93a6739805166d7ce2dfee85f24d45c16e WatchSource:0}: Error finding container e2c665cc3aa61787d35a53d97f5f7d93a6739805166d7ce2dfee85f24d45c16e: Status 404 returned error can't find the container with id e2c665cc3aa61787d35a53d97f5f7d93a6739805166d7ce2dfee85f24d45c16e Sep 17 19:07:56.098258 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:56.098229 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-tls\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:56.100729 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:56.100707 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e16b1bb-d157-4f46-85c4-6ce3e028886d-node-exporter-tls\") pod \"node-exporter-zrzgc\" (UID: \"7e16b1bb-d157-4f46-85c4-6ce3e028886d\") " pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:56.279873 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:56.279837 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zrzgc" Sep 17 19:07:56.292143 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:56.292094 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e16b1bb_d157_4f46_85c4_6ce3e028886d.slice/crio-abf7dcf49e214de6e9006df9c01ea7590dcdccfe392e53b98141b7e18ada450c WatchSource:0}: Error finding container abf7dcf49e214de6e9006df9c01ea7590dcdccfe392e53b98141b7e18ada450c: Status 404 returned error can't find the container with id abf7dcf49e214de6e9006df9c01ea7590dcdccfe392e53b98141b7e18ada450c Sep 17 19:07:56.643332 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:56.643241 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zrzgc" event={"ID":"7e16b1bb-d157-4f46-85c4-6ce3e028886d","Type":"ContainerStarted","Data":"abf7dcf49e214de6e9006df9c01ea7590dcdccfe392e53b98141b7e18ada450c"} Sep 17 19:07:56.644955 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:56.644927 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" event={"ID":"5842211a-e5f4-46a5-abdc-765ec50dcd98","Type":"ContainerStarted","Data":"e2c665cc3aa61787d35a53d97f5f7d93a6739805166d7ce2dfee85f24d45c16e"} Sep 17 19:07:57.346279 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.346255 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp"] Sep 17 19:07:57.349560 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.349543 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.353589 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.353353 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-53hr5r2sudlir\"" Sep 17 19:07:57.353589 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.353364 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 17 19:07:57.353589 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.353373 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8dzkb\"" Sep 17 19:07:57.353589 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.353459 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 17 19:07:57.353589 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.353521 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 17 19:07:57.353589 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.353536 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 17 19:07:57.353914 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.353620 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 17 19:07:57.366775 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.366752 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp"] Sep 17 19:07:57.513917 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.513486 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/457a7e56-d003-435f-b974-a2b77056babb-metrics-client-ca\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.513917 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.513555 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.513917 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.513592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-grpc-tls\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.513917 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.513620 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-tls\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.513917 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.513654 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjstq\" (UniqueName: \"kubernetes.io/projected/457a7e56-d003-435f-b974-a2b77056babb-kube-api-access-tjstq\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.513917 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.513690 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.513917 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.513744 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.513917 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.513779 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.614419 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.614392 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.614573 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.614425 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-grpc-tls\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.614573 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.614448 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-tls\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.614573 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.614470 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjstq\" (UniqueName: \"kubernetes.io/projected/457a7e56-d003-435f-b974-a2b77056babb-kube-api-access-tjstq\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.614573 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.614493 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.614573 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.614531 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.614573 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.614552 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.614890 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.614588 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/457a7e56-d003-435f-b974-a2b77056babb-metrics-client-ca\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.615410 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.615352 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/457a7e56-d003-435f-b974-a2b77056babb-metrics-client-ca\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.617627 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.617579 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-grpc-tls\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.617738 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.617716 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.617898 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.617863 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.617988 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.617920 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-tls\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.618030 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.618013 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.618462 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.618441 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/457a7e56-d003-435f-b974-a2b77056babb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.624160 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.624051 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjstq\" (UniqueName: \"kubernetes.io/projected/457a7e56-d003-435f-b974-a2b77056babb-kube-api-access-tjstq\") pod \"thanos-querier-7d99bc4c6f-grdmp\" (UID: \"457a7e56-d003-435f-b974-a2b77056babb\") " pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.632925 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.632900 2563 patch_prober.go:28] interesting pod/image-registry-758b8c48b6-7p8j9 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 17 19:07:57.633017 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.632949 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" podUID="2b58566f-f608-4e4d-9e2f-4bd88bb65202" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:07:57.651068 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.650908 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" event={"ID":"5842211a-e5f4-46a5-abdc-765ec50dcd98","Type":"ContainerStarted","Data":"d54a3f0c859fb8a5b137624dc5342ca226f247e405a0c2e5cadd2e820500f18c"} Sep 17 19:07:57.651068 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.650977 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" event={"ID":"5842211a-e5f4-46a5-abdc-765ec50dcd98","Type":"ContainerStarted","Data":"acfa366d964e1ca39700766c0a746fb966e5c5fef76fee2c269fb75a62d0688c"} Sep 17 19:07:57.651068 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.651005 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" event={"ID":"5842211a-e5f4-46a5-abdc-765ec50dcd98","Type":"ContainerStarted","Data":"0f9cb77292a2b3b30eb5129b81ddb7d98ccf2242a3d3552ebe195b9271c788e6"} Sep 17 19:07:57.653112 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.653071 2563 generic.go:356] "Generic (PLEG): container finished" podID="7e16b1bb-d157-4f46-85c4-6ce3e028886d" containerID="c9a3be3d567221a42149a7a5f5dc2d2d03566141617d919c9d6a316c5d1d2dec" exitCode=0 Sep 17 19:07:57.653217 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.653125 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zrzgc" event={"ID":"7e16b1bb-d157-4f46-85c4-6ce3e028886d","Type":"ContainerDied","Data":"c9a3be3d567221a42149a7a5f5dc2d2d03566141617d919c9d6a316c5d1d2dec"} Sep 17 19:07:57.659888 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.659869 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:07:57.671867 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.670629 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-qfpj5" podStartSLOduration=1.276528987 podStartE2EDuration="2.670607997s" totalImagesPullingTime="1.39407901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:57.669392253 +0000 UTC m=+85.931265661" watchObservedRunningTime="2026-09-17 19:07:57.670607997 +0000 UTC m=+85.932481406" Sep 17 19:07:57.954799 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:57.954727 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp"] Sep 17 19:07:57.958353 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:07:57.958330 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457a7e56_d003_435f_b974_a2b77056babb.slice/crio-673f7d32d20c02a3e8981e14eb823365d22e9083e4a87d67b5f12809d6be53cc WatchSource:0}: Error finding container 673f7d32d20c02a3e8981e14eb823365d22e9083e4a87d67b5f12809d6be53cc: Status 404 returned error can't find the container with id 673f7d32d20c02a3e8981e14eb823365d22e9083e4a87d67b5f12809d6be53cc Sep 17 19:07:58.659652 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:58.659611 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zrzgc" event={"ID":"7e16b1bb-d157-4f46-85c4-6ce3e028886d","Type":"ContainerStarted","Data":"052eca87408c690dc256922b2148864f414414c9425a6fe4498c02683a6c617d"} Sep 17 19:07:58.660155 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:58.659715 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zrzgc" event={"ID":"7e16b1bb-d157-4f46-85c4-6ce3e028886d","Type":"ContainerStarted","Data":"6d53e07413815ce3cccda1809b277eaa26d5c4bd41d0ae09d851e018f04156a9"} Sep 17 19:07:58.661138 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:58.661088 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" event={"ID":"457a7e56-d003-435f-b974-a2b77056babb","Type":"ContainerStarted","Data":"673f7d32d20c02a3e8981e14eb823365d22e9083e4a87d67b5f12809d6be53cc"} Sep 17 19:07:58.680797 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:58.680747 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zrzgc" podStartSLOduration=2.53612415 podStartE2EDuration="3.680731782s" totalImagesPullingTime="1.049600789s" totalInitContainerRuntime="95.006843ms" isStatefulPod=true podCreationTimestamp="2026-09-17 19:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:58.679908676 +0000 UTC m=+86.941782283" watchObservedRunningTime="2026-09-17 19:07:58.680731782 +0000 UTC m=+86.942605188" Sep 17 19:07:59.555691 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.555655 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:07:59.674504 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.674473 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-664fc8f54d-fclsg"] Sep 17 19:07:59.679388 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.679364 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.682708 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.682678 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 17 19:07:59.682838 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.682754 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 17 19:07:59.682838 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.682758 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8l7rushn15s4b\"" Sep 17 19:07:59.683233 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.683216 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 17 19:07:59.683318 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.683237 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 17 19:07:59.683318 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.683256 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-lc6tw\"" Sep 17 19:07:59.699292 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.699265 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-664fc8f54d-fclsg"] Sep 17 19:07:59.835223 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.835201 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2e737871-bcb4-4d88-bd9e-99176999cedf-audit-log\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.835326 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.835249 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2e737871-bcb4-4d88-bd9e-99176999cedf-secret-metrics-server-tls\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.835412 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.835387 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2e737871-bcb4-4d88-bd9e-99176999cedf-secret-metrics-server-client-certs\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.835460 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.835443 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2e737871-bcb4-4d88-bd9e-99176999cedf-client-ca-bundle\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.835520 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.835505 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e737871-bcb4-4d88-bd9e-99176999cedf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.835582 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.835564 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9g5g\" (UniqueName: \"kubernetes.io/projected/2e737871-bcb4-4d88-bd9e-99176999cedf-kube-api-access-w9g5g\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.835646 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.835592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2e737871-bcb4-4d88-bd9e-99176999cedf-metrics-server-audit-profiles\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.936339 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.936306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2e737871-bcb4-4d88-bd9e-99176999cedf-audit-log\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.936505 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.936351 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2e737871-bcb4-4d88-bd9e-99176999cedf-secret-metrics-server-tls\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.936505 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.936485 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2e737871-bcb4-4d88-bd9e-99176999cedf-secret-metrics-server-client-certs\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.936577 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.936529 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2e737871-bcb4-4d88-bd9e-99176999cedf-client-ca-bundle\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.936577 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.936567 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e737871-bcb4-4d88-bd9e-99176999cedf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.936669 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.936601 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9g5g\" (UniqueName: \"kubernetes.io/projected/2e737871-bcb4-4d88-bd9e-99176999cedf-kube-api-access-w9g5g\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.936669 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.936633 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2e737871-bcb4-4d88-bd9e-99176999cedf-metrics-server-audit-profiles\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.937330 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.937077 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2e737871-bcb4-4d88-bd9e-99176999cedf-audit-log\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.937675 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.937650 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e737871-bcb4-4d88-bd9e-99176999cedf-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.938013 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.937986 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2e737871-bcb4-4d88-bd9e-99176999cedf-metrics-server-audit-profiles\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.938952 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.938926 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2e737871-bcb4-4d88-bd9e-99176999cedf-secret-metrics-server-tls\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.939144 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.939126 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2e737871-bcb4-4d88-bd9e-99176999cedf-secret-metrics-server-client-certs\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.939340 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.939322 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2e737871-bcb4-4d88-bd9e-99176999cedf-client-ca-bundle\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.960202 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.960175 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9g5g\" (UniqueName: \"kubernetes.io/projected/2e737871-bcb4-4d88-bd9e-99176999cedf-kube-api-access-w9g5g\") pod \"metrics-server-664fc8f54d-fclsg\" (UID: \"2e737871-bcb4-4d88-bd9e-99176999cedf\") " pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:07:59.989970 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:07:59.989931 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:08:00.043884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.043773 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bf94b879-k9s9l"] Sep 17 19:08:00.048907 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.048781 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.051913 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.051722 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 17 19:08:00.052338 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.052317 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 17 19:08:00.053588 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.052660 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 17 19:08:00.053686 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.052754 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 17 19:08:00.053738 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.052872 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 17 19:08:00.053789 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.052958 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 17 19:08:00.053836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.053012 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 17 19:08:00.053878 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.053454 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qv7gn\"" Sep 17 19:08:00.068930 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.068865 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bf94b879-k9s9l"] Sep 17 19:08:00.070489 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.070394 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 17 19:08:00.138754 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.138722 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-service-ca\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.138952 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.138768 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ns8f\" (UniqueName: \"kubernetes.io/projected/43f274df-ca60-4ee2-861c-e109a1b1b832-kube-api-access-2ns8f\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.138952 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.138794 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-oauth-serving-cert\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.138952 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.138854 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-serving-cert\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.138952 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.138901 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-oauth-config\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.138952 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.138923 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-trusted-ca-bundle\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.138952 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.138945 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-console-config\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.240435 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.240400 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-oauth-serving-cert\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.240626 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.240441 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-serving-cert\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.240626 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.240468 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-oauth-config\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.240626 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.240490 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-trusted-ca-bundle\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.240626 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.240513 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-console-config\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.240626 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.240596 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-service-ca\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.240860 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.240637 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ns8f\" (UniqueName: \"kubernetes.io/projected/43f274df-ca60-4ee2-861c-e109a1b1b832-kube-api-access-2ns8f\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.241300 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.241222 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-oauth-serving-cert\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.241520 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.241354 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-console-config\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.241520 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.241384 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-service-ca\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.241639 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.241532 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-trusted-ca-bundle\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.243176 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.243157 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-serving-cert\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.243282 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.243183 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-oauth-config\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.249825 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.249797 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ns8f\" (UniqueName: \"kubernetes.io/projected/43f274df-ca60-4ee2-861c-e109a1b1b832-kube-api-access-2ns8f\") pod \"console-7bf94b879-k9s9l\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.271296 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.271266 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-664fc8f54d-fclsg"] Sep 17 19:08:00.274747 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:08:00.274723 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e737871_bcb4_4d88_bd9e_99176999cedf.slice/crio-c924158290988e694f6f241cb640cda6e0452ccad959b1754563b8ac25f211d6 WatchSource:0}: Error finding container c924158290988e694f6f241cb640cda6e0452ccad959b1754563b8ac25f211d6: Status 404 returned error can't find the container with id c924158290988e694f6f241cb640cda6e0452ccad959b1754563b8ac25f211d6 Sep 17 19:08:00.363724 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.363638 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:00.659627 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.659604 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bf94b879-k9s9l"] Sep 17 19:08:00.662792 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:08:00.662770 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43f274df_ca60_4ee2_861c_e109a1b1b832.slice/crio-7a4e8881e081dbf87cb93d14b1ac979cd395165dc88f400e06764d92d4b87cf7 WatchSource:0}: Error finding container 7a4e8881e081dbf87cb93d14b1ac979cd395165dc88f400e06764d92d4b87cf7: Status 404 returned error can't find the container with id 7a4e8881e081dbf87cb93d14b1ac979cd395165dc88f400e06764d92d4b87cf7 Sep 17 19:08:00.671699 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.671668 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" event={"ID":"457a7e56-d003-435f-b974-a2b77056babb","Type":"ContainerStarted","Data":"bac2766106d4a0acba9b00d25dcfe90c84c86e21649d5a5279bd7d0a3df6b538"} Sep 17 19:08:00.671965 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.671936 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" event={"ID":"457a7e56-d003-435f-b974-a2b77056babb","Type":"ContainerStarted","Data":"a70cfede2a2fb54e21992ca4c1c201ea0d43a8ecf6b87ce61b17ce68fff6a23e"} Sep 17 19:08:00.672061 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.671983 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" event={"ID":"457a7e56-d003-435f-b974-a2b77056babb","Type":"ContainerStarted","Data":"8a4d2cfbd7a83df9c2656664dc3b382e44b1b86bb5ed31cbe78f64572fc1bbfa"} Sep 17 19:08:00.673259 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.673234 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf94b879-k9s9l" event={"ID":"43f274df-ca60-4ee2-861c-e109a1b1b832","Type":"ContainerStarted","Data":"7a4e8881e081dbf87cb93d14b1ac979cd395165dc88f400e06764d92d4b87cf7"} Sep 17 19:08:00.674939 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:00.674912 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" event={"ID":"2e737871-bcb4-4d88-bd9e-99176999cedf","Type":"ContainerStarted","Data":"c924158290988e694f6f241cb640cda6e0452ccad959b1754563b8ac25f211d6"} Sep 17 19:08:01.590796 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.587069 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 19:08:01.595144 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.594964 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.597783 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.597608 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jjhqx\"" Sep 17 19:08:01.601439 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.600575 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 17 19:08:01.601439 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.600808 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 17 19:08:01.601439 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.600976 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 17 19:08:01.601439 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.601127 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 17 19:08:01.601439 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.601321 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 17 19:08:01.602129 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.602010 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 17 19:08:01.606217 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.602400 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 17 19:08:01.606217 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.602603 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6jk2dt19vastq\"" Sep 17 19:08:01.606217 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.602812 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 17 19:08:01.606217 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.603834 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 17 19:08:01.606217 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.604048 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 17 19:08:01.607262 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.607138 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 17 19:08:01.607564 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.607418 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 17 19:08:01.607564 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.607464 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 17 19:08:01.612738 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.610694 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 19:08:01.612738 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.611819 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 17 19:08:01.689447 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.689404 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" event={"ID":"457a7e56-d003-435f-b974-a2b77056babb","Type":"ContainerStarted","Data":"172c14512baf9c10baa6cc801d5343934be6ebc0f9562f5fd265923f133f80c5"} Sep 17 19:08:01.689897 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.689496 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" event={"ID":"457a7e56-d003-435f-b974-a2b77056babb","Type":"ContainerStarted","Data":"34cc08450abaf0fb4cf8bfd750587956f0df21d9eb72d0a5f94d668139db7cbb"} Sep 17 19:08:01.689897 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.689542 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" event={"ID":"457a7e56-d003-435f-b974-a2b77056babb","Type":"ContainerStarted","Data":"0690f4fd57aa7b230d82e8a8e35cae65d212ff84625398d16efdf191444a64da"} Sep 17 19:08:01.690187 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.690171 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:08:01.719511 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.719188 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" podStartSLOduration=2.26271318 podStartE2EDuration="4.719165897s" totalImagesPullingTime="2.456452717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:08:01.717389625 +0000 UTC m=+89.979263074" watchObservedRunningTime="2026-09-17 19:08:01.719165897 +0000 UTC m=+89.981039295" Sep 17 19:08:01.757508 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757473 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-config\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.757508 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757526 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.757783 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757559 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.757783 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757586 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.757783 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757608 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.757783 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757652 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.757783 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757692 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.757783 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757720 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmphj\" (UniqueName: \"kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-kube-api-access-xmphj\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.757783 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757748 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.757783 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757777 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.758275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757805 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-config-out\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.758275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757845 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.758275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.758275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757893 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.758275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757932 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.758275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757959 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.758275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.757981 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.758275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.758022 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.758275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.758046 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.758275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.758125 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-web-config\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.858679 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.858592 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.858679 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.858636 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.858913 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.858691 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-web-config\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.858913 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.858735 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-config\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.858913 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.858785 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.858913 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.858821 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.858913 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.858846 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.858913 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.858870 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.858913 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.858906 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.858967 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859009 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmphj\" (UniqueName: \"kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-kube-api-access-xmphj\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859053 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859133 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-config-out\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859213 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859271 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859239 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859607 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859263 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859607 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859324 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859607 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859351 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859607 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859376 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.859979 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859924 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.860526 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.859130 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.862064 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.862039 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.862500 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.862477 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.866128 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.865398 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.866128 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.865737 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.866128 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.865886 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.866128 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.866063 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.869030 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.869005 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.874526 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.874507 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.875850 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.875598 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.876585 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.875918 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-config-out\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.876714 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.876486 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.877673 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.877199 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-web-config\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.877673 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.877238 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmphj\" (UniqueName: \"kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-kube-api-access-xmphj\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.877834 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.877778 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.877834 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.877790 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.878407 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.878383 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.878858 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.878713 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.880138 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.880078 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-config\") pod \"prometheus-k8s-0\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:01.921009 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:01.920936 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:02.461884 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:02.461819 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 19:08:02.697913 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:02.697339 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" event={"ID":"2e737871-bcb4-4d88-bd9e-99176999cedf","Type":"ContainerStarted","Data":"549441bb3495be64ebdcd5617441026948e7ee4d676207be40ae36e4040ac230"} Sep 17 19:08:02.717971 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:02.717873 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" podStartSLOduration=1.7883125949999998 podStartE2EDuration="3.717856796s" totalImagesPullingTime="1.929544201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:08:02.716306173 +0000 UTC m=+90.978179576" watchObservedRunningTime="2026-09-17 19:08:02.717856796 +0000 UTC m=+90.979730248" Sep 17 19:08:03.260830 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:08:03.260797 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f55592a_8f21_4a6c_b634_32b150850732.slice/crio-9315338ae6424b81b19d8d799dae1cf69e46cfb09146f39a02d1daf68eef3e0e WatchSource:0}: Error finding container 9315338ae6424b81b19d8d799dae1cf69e46cfb09146f39a02d1daf68eef3e0e: Status 404 returned error can't find the container with id 9315338ae6424b81b19d8d799dae1cf69e46cfb09146f39a02d1daf68eef3e0e Sep 17 19:08:03.703007 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:03.702925 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerStarted","Data":"9315338ae6424b81b19d8d799dae1cf69e46cfb09146f39a02d1daf68eef3e0e"} Sep 17 19:08:05.713619 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:05.713585 2563 generic.go:356] "Generic (PLEG): container finished" podID="6f55592a-8f21-4a6c-b634-32b150850732" containerID="e0cbbfe4231da1cd678bb34e049f77c84df0246537913081295a19e1e4e6bb3a" exitCode=0 Sep 17 19:08:05.713976 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:05.713659 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerDied","Data":"e0cbbfe4231da1cd678bb34e049f77c84df0246537913081295a19e1e4e6bb3a"} Sep 17 19:08:05.715643 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:05.715623 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf94b879-k9s9l" event={"ID":"43f274df-ca60-4ee2-861c-e109a1b1b832","Type":"ContainerStarted","Data":"0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b"} Sep 17 19:08:05.768483 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:05.768426 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bf94b879-k9s9l" podStartSLOduration=1.380067223 podStartE2EDuration="5.768407886s" totalImagesPullingTime="4.388340663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:08:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:08:05.767006528 +0000 UTC m=+94.028879949" watchObservedRunningTime="2026-09-17 19:08:05.768407886 +0000 UTC m=+94.030281293" Sep 17 19:08:07.705244 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:07.705207 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d99bc4c6f-grdmp" Sep 17 19:08:08.732165 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:08.732131 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerStarted","Data":"c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e"} Sep 17 19:08:09.739463 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:09.739376 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerStarted","Data":"e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c"} Sep 17 19:08:09.739812 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:09.739476 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerStarted","Data":"4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8"} Sep 17 19:08:09.739812 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:09.739503 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerStarted","Data":"c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731"} Sep 17 19:08:09.739812 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:09.739528 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerStarted","Data":"a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7"} Sep 17 19:08:09.739812 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:09.739551 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerStarted","Data":"cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2"} Sep 17 19:08:09.773957 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:09.773909 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.538026196 podStartE2EDuration="8.773890852s" totalImagesPullingTime="5.216846101s" totalInitContainerRuntime="19.018555ms" isStatefulPod=false podCreationTimestamp="2026-09-17 19:08:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:08:09.771041225 +0000 UTC m=+98.032914650" watchObservedRunningTime="2026-09-17 19:08:09.773890852 +0000 UTC m=+98.035764259" Sep 17 19:08:09.885739 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:09.885702 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-758b8c48b6-7p8j9"] Sep 17 19:08:10.131937 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:10.131851 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bf94b879-k9s9l"] Sep 17 19:08:10.364296 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:10.364263 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:11.922085 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:11.922054 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:08:14.588317 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:14.588289 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z7q6d" Sep 17 19:08:19.991040 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:19.991003 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:08:19.991539 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:19.991072 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:08:34.905903 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:34.905824 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" podUID="2b58566f-f608-4e4d-9e2f-4bd88bb65202" containerName="registry" containerID="cri-o://c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e" gracePeriod=30 Sep 17 19:08:35.152254 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.152191 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bf94b879-k9s9l" podUID="43f274df-ca60-4ee2-861c-e109a1b1b832" containerName="console" containerID="cri-o://0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b" gracePeriod=15 Sep 17 19:08:35.154389 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.154365 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:08:35.261164 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.261133 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls\") pod \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " Sep 17 19:08:35.261334 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.261192 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-image-registry-private-configuration\") pod \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " Sep 17 19:08:35.261334 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.261224 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j77jt\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-kube-api-access-j77jt\") pod \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " Sep 17 19:08:35.261334 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.261272 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-bound-sa-token\") pod \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " Sep 17 19:08:35.261334 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.261300 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2b58566f-f608-4e4d-9e2f-4bd88bb65202-ca-trust-extracted\") pod \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " Sep 17 19:08:35.261334 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.261324 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-certificates\") pod \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " Sep 17 19:08:35.261578 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.261401 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-installation-pull-secrets\") pod \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " Sep 17 19:08:35.261578 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.261424 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-trusted-ca\") pod \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\" (UID: \"2b58566f-f608-4e4d-9e2f-4bd88bb65202\") " Sep 17 19:08:35.262303 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.262271 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2b58566f-f608-4e4d-9e2f-4bd88bb65202" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:08:35.262634 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.262596 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2b58566f-f608-4e4d-9e2f-4bd88bb65202" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:08:35.264281 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.264193 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2b58566f-f608-4e4d-9e2f-4bd88bb65202" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:08:35.264281 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.264247 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2b58566f-f608-4e4d-9e2f-4bd88bb65202" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:08:35.264493 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.264307 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-kube-api-access-j77jt" (OuterVolumeSpecName: "kube-api-access-j77jt") pod "2b58566f-f608-4e4d-9e2f-4bd88bb65202" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202"). InnerVolumeSpecName "kube-api-access-j77jt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:08:35.264551 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.264498 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2b58566f-f608-4e4d-9e2f-4bd88bb65202" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:08:35.264649 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.264625 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2b58566f-f608-4e4d-9e2f-4bd88bb65202" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:08:35.271230 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.271185 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b58566f-f608-4e4d-9e2f-4bd88bb65202-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2b58566f-f608-4e4d-9e2f-4bd88bb65202" (UID: "2b58566f-f608-4e4d-9e2f-4bd88bb65202"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 19:08:35.362824 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.362789 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-installation-pull-secrets\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.362824 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.362821 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-trusted-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.362824 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.362831 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.363170 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.362840 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2b58566f-f608-4e4d-9e2f-4bd88bb65202-image-registry-private-configuration\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.363170 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.362851 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j77jt\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-kube-api-access-j77jt\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.363170 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.362860 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b58566f-f608-4e4d-9e2f-4bd88bb65202-bound-sa-token\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.363170 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.362868 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2b58566f-f608-4e4d-9e2f-4bd88bb65202-ca-trust-extracted\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.363170 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.362876 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2b58566f-f608-4e4d-9e2f-4bd88bb65202-registry-certificates\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.385201 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.385175 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:35.464159 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464121 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-service-ca\") pod \"43f274df-ca60-4ee2-861c-e109a1b1b832\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " Sep 17 19:08:35.464318 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464167 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ns8f\" (UniqueName: \"kubernetes.io/projected/43f274df-ca60-4ee2-861c-e109a1b1b832-kube-api-access-2ns8f\") pod \"43f274df-ca60-4ee2-861c-e109a1b1b832\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " Sep 17 19:08:35.464318 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464183 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-console-config\") pod \"43f274df-ca60-4ee2-861c-e109a1b1b832\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " Sep 17 19:08:35.464318 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464288 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-serving-cert\") pod \"43f274df-ca60-4ee2-861c-e109a1b1b832\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " Sep 17 19:08:35.464434 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464375 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-oauth-config\") pod \"43f274df-ca60-4ee2-861c-e109a1b1b832\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " Sep 17 19:08:35.464434 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464415 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-trusted-ca-bundle\") pod \"43f274df-ca60-4ee2-861c-e109a1b1b832\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " Sep 17 19:08:35.464511 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464477 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-oauth-serving-cert\") pod \"43f274df-ca60-4ee2-861c-e109a1b1b832\" (UID: \"43f274df-ca60-4ee2-861c-e109a1b1b832\") " Sep 17 19:08:35.464561 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464532 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-service-ca" (OuterVolumeSpecName: "service-ca") pod "43f274df-ca60-4ee2-861c-e109a1b1b832" (UID: "43f274df-ca60-4ee2-861c-e109a1b1b832"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:08:35.464617 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464570 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-console-config" (OuterVolumeSpecName: "console-config") pod "43f274df-ca60-4ee2-861c-e109a1b1b832" (UID: "43f274df-ca60-4ee2-861c-e109a1b1b832"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:08:35.464856 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464823 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.464856 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464848 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-console-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.464995 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464878 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "43f274df-ca60-4ee2-861c-e109a1b1b832" (UID: "43f274df-ca60-4ee2-861c-e109a1b1b832"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:08:35.464995 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.464928 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "43f274df-ca60-4ee2-861c-e109a1b1b832" (UID: "43f274df-ca60-4ee2-861c-e109a1b1b832"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:08:35.466762 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.466739 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43f274df-ca60-4ee2-861c-e109a1b1b832-kube-api-access-2ns8f" (OuterVolumeSpecName: "kube-api-access-2ns8f") pod "43f274df-ca60-4ee2-861c-e109a1b1b832" (UID: "43f274df-ca60-4ee2-861c-e109a1b1b832"). InnerVolumeSpecName "kube-api-access-2ns8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:08:35.466862 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.466779 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "43f274df-ca60-4ee2-861c-e109a1b1b832" (UID: "43f274df-ca60-4ee2-861c-e109a1b1b832"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:08:35.466862 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.466796 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "43f274df-ca60-4ee2-861c-e109a1b1b832" (UID: "43f274df-ca60-4ee2-861c-e109a1b1b832"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:08:35.566156 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.566083 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-trusted-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.566156 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.566152 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43f274df-ca60-4ee2-861c-e109a1b1b832-oauth-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.566156 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.566162 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ns8f\" (UniqueName: \"kubernetes.io/projected/43f274df-ca60-4ee2-861c-e109a1b1b832-kube-api-access-2ns8f\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.566156 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.566178 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.566462 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.566188 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43f274df-ca60-4ee2-861c-e109a1b1b832-console-oauth-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:08:35.849328 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.849230 2563 generic.go:356] "Generic (PLEG): container finished" podID="2b58566f-f608-4e4d-9e2f-4bd88bb65202" containerID="c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e" exitCode=0 Sep 17 19:08:35.849328 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.849301 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" Sep 17 19:08:35.849533 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.849322 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" event={"ID":"2b58566f-f608-4e4d-9e2f-4bd88bb65202","Type":"ContainerDied","Data":"c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e"} Sep 17 19:08:35.849533 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.849381 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758b8c48b6-7p8j9" event={"ID":"2b58566f-f608-4e4d-9e2f-4bd88bb65202","Type":"ContainerDied","Data":"84ee2faaf1c470763d0bb839dcfae16f3adaa15f8f2adb47d523b10be2af1135"} Sep 17 19:08:35.849533 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.849407 2563 scope.go:117] "RemoveContainer" containerID="c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e" Sep 17 19:08:35.850787 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.850761 2563 generic.go:356] "Generic (PLEG): container finished" podID="43f274df-ca60-4ee2-861c-e109a1b1b832" containerID="0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b" exitCode=0 Sep 17 19:08:35.850897 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.850804 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf94b879-k9s9l" event={"ID":"43f274df-ca60-4ee2-861c-e109a1b1b832","Type":"ContainerDied","Data":"0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b"} Sep 17 19:08:35.850897 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.850823 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bf94b879-k9s9l" Sep 17 19:08:35.850897 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.850841 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bf94b879-k9s9l" event={"ID":"43f274df-ca60-4ee2-861c-e109a1b1b832","Type":"ContainerDied","Data":"7a4e8881e081dbf87cb93d14b1ac979cd395165dc88f400e06764d92d4b87cf7"} Sep 17 19:08:35.858719 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.858691 2563 scope.go:117] "RemoveContainer" containerID="c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e" Sep 17 19:08:35.859128 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:08:35.859066 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e\": container with ID starting with c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e not found: ID does not exist" containerID="c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e" Sep 17 19:08:35.859226 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.859125 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e"} err="failed to get container status \"c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e\": rpc error: code = NotFound desc = could not find container \"c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e\": container with ID starting with c37e1917e83b94cb1bb4d69c6fe4a4233f796cade0c0e52330f3933711324e3e not found: ID does not exist" Sep 17 19:08:35.859226 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.859157 2563 scope.go:117] "RemoveContainer" containerID="0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b" Sep 17 19:08:35.867852 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.867835 2563 scope.go:117] "RemoveContainer" containerID="0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b" Sep 17 19:08:35.868164 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:08:35.868144 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b\": container with ID starting with 0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b not found: ID does not exist" containerID="0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b" Sep 17 19:08:35.868228 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.868174 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b"} err="failed to get container status \"0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b\": rpc error: code = NotFound desc = could not find container \"0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b\": container with ID starting with 0b3df66bd0a38bd803f9c7e299b67c66cd38a97202331bb902e27b66360b1d8b not found: ID does not exist" Sep 17 19:08:35.875250 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.875210 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-758b8c48b6-7p8j9"] Sep 17 19:08:35.881429 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.881404 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-758b8c48b6-7p8j9"] Sep 17 19:08:35.891730 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.891702 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bf94b879-k9s9l"] Sep 17 19:08:35.897317 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:35.897291 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bf94b879-k9s9l"] Sep 17 19:08:36.234532 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:36.234498 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b58566f-f608-4e4d-9e2f-4bd88bb65202" path="/var/lib/kubelet/pods/2b58566f-f608-4e4d-9e2f-4bd88bb65202/volumes" Sep 17 19:08:36.234898 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:36.234881 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43f274df-ca60-4ee2-861c-e109a1b1b832" path="/var/lib/kubelet/pods/43f274df-ca60-4ee2-861c-e109a1b1b832/volumes" Sep 17 19:08:39.995727 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:39.995697 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:08:39.999817 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:39.999793 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-664fc8f54d-fclsg" Sep 17 19:08:43.889043 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:43.889006 2563 generic.go:356] "Generic (PLEG): container finished" podID="0280dd0f-999f-44b2-b052-4652b01c394c" containerID="9e695da033557cc6c8a16351b5ffe5cb31811ec0fe88bbf3e3b485b2f0f8a592" exitCode=0 Sep 17 19:08:43.889503 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:43.889080 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" event={"ID":"0280dd0f-999f-44b2-b052-4652b01c394c","Type":"ContainerDied","Data":"9e695da033557cc6c8a16351b5ffe5cb31811ec0fe88bbf3e3b485b2f0f8a592"} Sep 17 19:08:43.889503 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:43.889449 2563 scope.go:117] "RemoveContainer" containerID="9e695da033557cc6c8a16351b5ffe5cb31811ec0fe88bbf3e3b485b2f0f8a592" Sep 17 19:08:44.895719 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:08:44.895682 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-mml6g" event={"ID":"0280dd0f-999f-44b2-b052-4652b01c394c","Type":"ContainerStarted","Data":"ae5a8a36008ef38a383fc34b6a059c23f9487c3f15756cd354d6b51c740153f7"} Sep 17 19:09:01.922065 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:01.922021 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:01.938966 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:01.938939 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:01.988744 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:01.988717 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:19.999337 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:19.999292 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 19:09:20.000270 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:20.000062 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="prometheus" containerID="cri-o://c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e" gracePeriod=600 Sep 17 19:09:20.000270 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:20.000117 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy" containerID="cri-o://4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8" gracePeriod=600 Sep 17 19:09:20.000270 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:20.000169 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="thanos-sidecar" containerID="cri-o://a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7" gracePeriod=600 Sep 17 19:09:20.000270 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:20.000185 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy-web" containerID="cri-o://c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731" gracePeriod=600 Sep 17 19:09:20.000270 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:20.000222 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c" gracePeriod=600 Sep 17 19:09:20.000622 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:20.000205 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="config-reloader" containerID="cri-o://cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2" gracePeriod=600 Sep 17 19:09:21.057778 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.057742 2563 generic.go:356] "Generic (PLEG): container finished" podID="6f55592a-8f21-4a6c-b634-32b150850732" containerID="e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c" exitCode=0 Sep 17 19:09:21.057778 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.057776 2563 generic.go:356] "Generic (PLEG): container finished" podID="6f55592a-8f21-4a6c-b634-32b150850732" containerID="4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8" exitCode=0 Sep 17 19:09:21.058275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.057785 2563 generic.go:356] "Generic (PLEG): container finished" podID="6f55592a-8f21-4a6c-b634-32b150850732" containerID="a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7" exitCode=0 Sep 17 19:09:21.058275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.057793 2563 generic.go:356] "Generic (PLEG): container finished" podID="6f55592a-8f21-4a6c-b634-32b150850732" containerID="cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2" exitCode=0 Sep 17 19:09:21.058275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.057802 2563 generic.go:356] "Generic (PLEG): container finished" podID="6f55592a-8f21-4a6c-b634-32b150850732" containerID="c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e" exitCode=0 Sep 17 19:09:21.058275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.057815 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerDied","Data":"e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c"} Sep 17 19:09:21.058275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.057917 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerDied","Data":"4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8"} Sep 17 19:09:21.058275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.057956 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerDied","Data":"a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7"} Sep 17 19:09:21.058275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.057996 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerDied","Data":"cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2"} Sep 17 19:09:21.058275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.058029 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerDied","Data":"c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e"} Sep 17 19:09:21.262116 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.262078 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:21.377825 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.377715 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-metrics-client-ca\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.377825 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.377764 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-db\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.377825 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.377818 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378132 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.377846 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-2\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378132 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.377869 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-metrics-client-certs\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378132 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.377890 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-web-config\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378132 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.377913 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-kubelet-serving-ca-bundle\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378132 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.377935 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-thanos-prometheus-http-client-file\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378132 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.377971 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378132 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.377999 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-tls-assets\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378132 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378020 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-0\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378132 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378078 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmphj\" (UniqueName: \"kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-kube-api-access-xmphj\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378603 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378138 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-tls\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378603 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378194 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-config\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378603 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378224 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-trusted-ca-bundle\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378603 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378237 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:09:21.378603 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378248 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:09:21.378603 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378262 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-1\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378603 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378319 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-serving-certs-ca-bundle\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378603 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378353 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-config-out\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378603 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378384 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-grpc-tls\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.378603 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378411 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-kube-rbac-proxy\") pod \"6f55592a-8f21-4a6c-b634-32b150850732\" (UID: \"6f55592a-8f21-4a6c-b634-32b150850732\") " Sep 17 19:09:21.379303 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378869 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-metrics-client-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.379303 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.378890 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.379403 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.379338 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 19:09:21.380456 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.379439 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:09:21.380577 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.380517 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:09:21.380674 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.380646 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:09:21.380910 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.380887 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:09:21.381530 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.381397 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:09:21.381852 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.381830 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:09:21.383295 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.383249 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-config-out" (OuterVolumeSpecName: "config-out") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 19:09:21.383418 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.383335 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:09:21.383708 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.383668 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:09:21.384033 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.384006 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:09:21.384176 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.384122 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-kube-api-access-xmphj" (OuterVolumeSpecName: "kube-api-access-xmphj") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "kube-api-access-xmphj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:09:21.384458 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.384427 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:09:21.387171 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.384877 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:09:21.387171 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.385302 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-config" (OuterVolumeSpecName: "config") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:09:21.391465 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.391431 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:09:21.392193 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.392168 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:09:21.399854 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.399829 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-web-config" (OuterVolumeSpecName: "web-config") pod "6f55592a-8f21-4a6c-b634-32b150850732" (UID: "6f55592a-8f21-4a6c-b634-32b150850732"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:09:21.479479 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479425 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479479 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479474 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479479 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479485 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-config-out\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479479 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479495 2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-grpc-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479505 2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-kube-rbac-proxy\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479514 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-db\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479523 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479533 2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-metrics-client-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479565 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-web-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479574 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479583 2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479591 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479599 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-tls-assets\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479607 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479615 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmphj\" (UniqueName: \"kubernetes.io/projected/6f55592a-8f21-4a6c-b634-32b150850732-kube-api-access-xmphj\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479623 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479631 2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/6f55592a-8f21-4a6c-b634-32b150850732-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:21.479717 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:21.479639 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f55592a-8f21-4a6c-b634-32b150850732-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:09:22.066192 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.066151 2563 generic.go:356] "Generic (PLEG): container finished" podID="6f55592a-8f21-4a6c-b634-32b150850732" containerID="c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731" exitCode=0 Sep 17 19:09:22.066510 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.066214 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerDied","Data":"c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731"} Sep 17 19:09:22.066510 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.066249 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.066510 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.066307 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f55592a-8f21-4a6c-b634-32b150850732","Type":"ContainerDied","Data":"9315338ae6424b81b19d8d799dae1cf69e46cfb09146f39a02d1daf68eef3e0e"} Sep 17 19:09:22.066510 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.066343 2563 scope.go:117] "RemoveContainer" containerID="e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c" Sep 17 19:09:22.074879 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.074855 2563 scope.go:117] "RemoveContainer" containerID="4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8" Sep 17 19:09:22.082970 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.082947 2563 scope.go:117] "RemoveContainer" containerID="c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731" Sep 17 19:09:22.090361 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.090333 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 19:09:22.090955 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.090940 2563 scope.go:117] "RemoveContainer" containerID="a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7" Sep 17 19:09:22.097330 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.097304 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 19:09:22.100391 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.100374 2563 scope.go:117] "RemoveContainer" containerID="cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2" Sep 17 19:09:22.108006 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.107981 2563 scope.go:117] "RemoveContainer" containerID="c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e" Sep 17 19:09:22.116179 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.116160 2563 scope.go:117] "RemoveContainer" containerID="e0cbbfe4231da1cd678bb34e049f77c84df0246537913081295a19e1e4e6bb3a" Sep 17 19:09:22.123989 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.123970 2563 scope.go:117] "RemoveContainer" containerID="e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c" Sep 17 19:09:22.124313 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:09:22.124293 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c\": container with ID starting with e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c not found: ID does not exist" containerID="e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c" Sep 17 19:09:22.124356 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.124324 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c"} err="failed to get container status \"e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c\": rpc error: code = NotFound desc = could not find container \"e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c\": container with ID starting with e8968c0a5839b8de6c923a6af512aa140e1332afb05f1230329fb4c8716cff4c not found: ID does not exist" Sep 17 19:09:22.124356 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.124346 2563 scope.go:117] "RemoveContainer" containerID="4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8" Sep 17 19:09:22.124642 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:09:22.124621 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8\": container with ID starting with 4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8 not found: ID does not exist" containerID="4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8" Sep 17 19:09:22.124674 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.124650 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8"} err="failed to get container status \"4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8\": rpc error: code = NotFound desc = could not find container \"4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8\": container with ID starting with 4fc3cdbdc46503b00599e0feaefedb45b173ca87b5f0dd664345ed69fb777ff8 not found: ID does not exist" Sep 17 19:09:22.124674 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.124665 2563 scope.go:117] "RemoveContainer" containerID="c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731" Sep 17 19:09:22.124894 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:09:22.124879 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731\": container with ID starting with c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731 not found: ID does not exist" containerID="c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731" Sep 17 19:09:22.124930 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.124899 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731"} err="failed to get container status \"c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731\": rpc error: code = NotFound desc = could not find container \"c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731\": container with ID starting with c5bbefd566668adeb97b4594e4cba3b6cb3cec6556995d776538aacce4f51731 not found: ID does not exist" Sep 17 19:09:22.124930 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.124912 2563 scope.go:117] "RemoveContainer" containerID="a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7" Sep 17 19:09:22.125178 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:09:22.125162 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7\": container with ID starting with a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7 not found: ID does not exist" containerID="a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7" Sep 17 19:09:22.125224 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.125197 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7"} err="failed to get container status \"a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7\": rpc error: code = NotFound desc = could not find container \"a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7\": container with ID starting with a899eb440819a557da754e8361ef60e114afc63252cdcfca1d4e0523bfed27b7 not found: ID does not exist" Sep 17 19:09:22.125224 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.125211 2563 scope.go:117] "RemoveContainer" containerID="cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2" Sep 17 19:09:22.125444 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:09:22.125426 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2\": container with ID starting with cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2 not found: ID does not exist" containerID="cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2" Sep 17 19:09:22.125492 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.125449 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2"} err="failed to get container status \"cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2\": rpc error: code = NotFound desc = could not find container \"cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2\": container with ID starting with cbea0092a84b068112b98d61430103bf37230bfc145953b2cf8c49a34bc6d7a2 not found: ID does not exist" Sep 17 19:09:22.125492 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.125463 2563 scope.go:117] "RemoveContainer" containerID="c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e" Sep 17 19:09:22.125693 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:09:22.125677 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e\": container with ID starting with c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e not found: ID does not exist" containerID="c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e" Sep 17 19:09:22.125726 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.125699 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e"} err="failed to get container status \"c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e\": rpc error: code = NotFound desc = could not find container \"c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e\": container with ID starting with c111a7cbcb50ae65b5b94391d341e7045ac6e1ecb67bc47d0e341e0f015dab8e not found: ID does not exist" Sep 17 19:09:22.125726 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.125712 2563 scope.go:117] "RemoveContainer" containerID="e0cbbfe4231da1cd678bb34e049f77c84df0246537913081295a19e1e4e6bb3a" Sep 17 19:09:22.125927 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:09:22.125912 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0cbbfe4231da1cd678bb34e049f77c84df0246537913081295a19e1e4e6bb3a\": container with ID starting with e0cbbfe4231da1cd678bb34e049f77c84df0246537913081295a19e1e4e6bb3a not found: ID does not exist" containerID="e0cbbfe4231da1cd678bb34e049f77c84df0246537913081295a19e1e4e6bb3a" Sep 17 19:09:22.125968 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.125930 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0cbbfe4231da1cd678bb34e049f77c84df0246537913081295a19e1e4e6bb3a"} err="failed to get container status \"e0cbbfe4231da1cd678bb34e049f77c84df0246537913081295a19e1e4e6bb3a\": rpc error: code = NotFound desc = could not find container \"e0cbbfe4231da1cd678bb34e049f77c84df0246537913081295a19e1e4e6bb3a\": container with ID starting with e0cbbfe4231da1cd678bb34e049f77c84df0246537913081295a19e1e4e6bb3a not found: ID does not exist" Sep 17 19:09:22.132772 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.132747 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 19:09:22.133202 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133182 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="config-reloader" Sep 17 19:09:22.133262 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133206 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="config-reloader" Sep 17 19:09:22.133262 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133219 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="thanos-sidecar" Sep 17 19:09:22.133262 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133227 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="thanos-sidecar" Sep 17 19:09:22.133262 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133242 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43f274df-ca60-4ee2-861c-e109a1b1b832" containerName="console" Sep 17 19:09:22.133262 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133250 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="43f274df-ca60-4ee2-861c-e109a1b1b832" containerName="console" Sep 17 19:09:22.133262 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133259 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy-thanos" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133267 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy-thanos" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133281 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b58566f-f608-4e4d-9e2f-4bd88bb65202" containerName="registry" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133288 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b58566f-f608-4e4d-9e2f-4bd88bb65202" containerName="registry" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133297 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="prometheus" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133305 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="prometheus" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133316 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="init-config-reloader" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133323 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="init-config-reloader" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133345 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy-web" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133353 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy-web" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133361 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133368 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy" Sep 17 19:09:22.133459 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133458 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy-thanos" Sep 17 19:09:22.133819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133470 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="43f274df-ca60-4ee2-861c-e109a1b1b832" containerName="console" Sep 17 19:09:22.133819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133479 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="config-reloader" Sep 17 19:09:22.133819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133487 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy" Sep 17 19:09:22.133819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133497 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b58566f-f608-4e4d-9e2f-4bd88bb65202" containerName="registry" Sep 17 19:09:22.133819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133505 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="prometheus" Sep 17 19:09:22.133819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133513 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="thanos-sidecar" Sep 17 19:09:22.133819 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.133521 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f55592a-8f21-4a6c-b634-32b150850732" containerName="kube-rbac-proxy-web" Sep 17 19:09:22.137070 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.137051 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.139529 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.139503 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 17 19:09:22.139784 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.139766 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 17 19:09:22.139867 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.139804 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 17 19:09:22.139867 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.139855 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 17 19:09:22.139966 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.139879 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 17 19:09:22.140063 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.140044 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jjhqx\"" Sep 17 19:09:22.140222 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.140199 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 17 19:09:22.140222 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.140217 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6jk2dt19vastq\"" Sep 17 19:09:22.140461 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.140254 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 17 19:09:22.140461 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.140344 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 17 19:09:22.140599 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.140582 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 17 19:09:22.140668 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.140658 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 17 19:09:22.140728 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.140658 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 17 19:09:22.140906 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.140891 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 17 19:09:22.146030 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.146000 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 17 19:09:22.149316 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.149291 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 17 19:09:22.156376 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.156339 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 19:09:22.234012 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.233977 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f55592a-8f21-4a6c-b634-32b150850732" path="/var/lib/kubelet/pods/6f55592a-8f21-4a6c-b634-32b150850732/volumes" Sep 17 19:09:22.286951 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.286921 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287126 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.286960 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287126 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.286979 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgdhk\" (UniqueName: \"kubernetes.io/projected/560326e5-d224-4110-9b53-e8bbaf2ddace-kube-api-access-kgdhk\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287126 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287027 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/560326e5-d224-4110-9b53-e8bbaf2ddace-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287126 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287049 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-web-config\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287126 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287067 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287126 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287084 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287126 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287124 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-config\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287367 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287171 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287367 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287206 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287367 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287238 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/560326e5-d224-4110-9b53-e8bbaf2ddace-config-out\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287367 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287274 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287367 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287305 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287367 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287321 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287367 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287367 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287351 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287639 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287369 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287639 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287639 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287451 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.287639 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.287477 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388246 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388144 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/560326e5-d224-4110-9b53-e8bbaf2ddace-config-out\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388246 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388185 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388455 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388275 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388455 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388308 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388455 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388324 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388455 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388341 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388455 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388363 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388455 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388397 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388816 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388515 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388816 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388578 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388816 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388616 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388816 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388649 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388816 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388689 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgdhk\" (UniqueName: \"kubernetes.io/projected/560326e5-d224-4110-9b53-e8bbaf2ddace-kube-api-access-kgdhk\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388816 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388765 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/560326e5-d224-4110-9b53-e8bbaf2ddace-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.388816 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388801 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-web-config\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.389207 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388833 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.389207 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388858 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.389207 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388901 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-config\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.389207 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388927 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.389207 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.388999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.389207 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.389003 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.389207 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.389071 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.389724 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.389621 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.389843 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.389732 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.389907 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.389881 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.392780 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.390941 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.392780 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.391474 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.392780 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.391807 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.392780 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.392718 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.392780 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.392735 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.393607 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.393582 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-config\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.393777 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.393752 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/560326e5-d224-4110-9b53-e8bbaf2ddace-config-out\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.393979 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.393937 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.394436 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.394405 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.394765 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.394743 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-web-config\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.394851 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.394779 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.395295 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.395278 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/560326e5-d224-4110-9b53-e8bbaf2ddace-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.396135 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.396116 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/560326e5-d224-4110-9b53-e8bbaf2ddace-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.396272 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.396253 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/560326e5-d224-4110-9b53-e8bbaf2ddace-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.401629 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.401604 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgdhk\" (UniqueName: \"kubernetes.io/projected/560326e5-d224-4110-9b53-e8bbaf2ddace-kube-api-access-kgdhk\") pod \"prometheus-k8s-0\" (UID: \"560326e5-d224-4110-9b53-e8bbaf2ddace\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.448733 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.448688 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:09:22.741143 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:22.741112 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 19:09:22.742438 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:09:22.742413 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod560326e5_d224_4110_9b53_e8bbaf2ddace.slice/crio-92eeca209c419822068d48e83d31f807f5245f781a502a9b838f17f655925dff WatchSource:0}: Error finding container 92eeca209c419822068d48e83d31f807f5245f781a502a9b838f17f655925dff: Status 404 returned error can't find the container with id 92eeca209c419822068d48e83d31f807f5245f781a502a9b838f17f655925dff Sep 17 19:09:23.072239 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:23.072134 2563 generic.go:356] "Generic (PLEG): container finished" podID="560326e5-d224-4110-9b53-e8bbaf2ddace" containerID="391c1eecc072d5ec028212f4b20899245ccf7effd221e4e0baee103396d607c7" exitCode=0 Sep 17 19:09:23.072632 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:23.072257 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"560326e5-d224-4110-9b53-e8bbaf2ddace","Type":"ContainerDied","Data":"391c1eecc072d5ec028212f4b20899245ccf7effd221e4e0baee103396d607c7"} Sep 17 19:09:23.072632 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:23.072336 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"560326e5-d224-4110-9b53-e8bbaf2ddace","Type":"ContainerStarted","Data":"92eeca209c419822068d48e83d31f807f5245f781a502a9b838f17f655925dff"} Sep 17 19:09:24.081159 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:24.081093 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"560326e5-d224-4110-9b53-e8bbaf2ddace","Type":"ContainerStarted","Data":"bef4ff318d612a107f4e5d5eda1a8b6f3589c9ee8417859022620197f1fcf6cd"} Sep 17 19:09:24.081520 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:24.081206 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"560326e5-d224-4110-9b53-e8bbaf2ddace","Type":"ContainerStarted","Data":"eec71bf0404f1dc388b4e9f2c39fe515dd570a1b579f1eb74f07b93f61158249"} Sep 17 19:09:24.081520 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:24.081253 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"560326e5-d224-4110-9b53-e8bbaf2ddace","Type":"ContainerStarted","Data":"7753b34d6ce1c2c8eaa4a587190e2b909bf2744d8de451bfd7cc440a3c9ee2a2"} Sep 17 19:09:24.081520 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:24.081291 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"560326e5-d224-4110-9b53-e8bbaf2ddace","Type":"ContainerStarted","Data":"9da3636fc7edb881d49175f452acdef59b5c73f213387379ad342ded88ab2e5a"} Sep 17 19:09:24.081520 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:24.081326 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"560326e5-d224-4110-9b53-e8bbaf2ddace","Type":"ContainerStarted","Data":"4bde807a589d01796f476a217db89b709f4ea8db3c5ccbd71c2897f1fe42a502"} Sep 17 19:09:24.081520 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:24.081361 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"560326e5-d224-4110-9b53-e8bbaf2ddace","Type":"ContainerStarted","Data":"77661f471487c4d13f77aa14400e86646c03634c2f702c5e0bc6e89d9307af5a"} Sep 17 19:09:24.110678 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:24.110612 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.086607631 podStartE2EDuration="2.11059317s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.985539ms" isStatefulPod=false podCreationTimestamp="2026-09-17 19:09:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:09:24.108342634 +0000 UTC m=+172.370216060" watchObservedRunningTime="2026-09-17 19:09:24.11059317 +0000 UTC m=+172.372466577" Sep 17 19:09:27.448877 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:09:27.448822 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:10:22.449948 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:10:22.449910 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:10:22.466681 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:10:22.466651 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:10:23.346256 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:10:23.346228 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 19:11:32.104665 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:11:32.104636 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:11:32.110614 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:11:32.110576 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:11:32.116359 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:11:32.116329 2563 kubelet.go:1648] "Image garbage collection succeeded" Sep 17 19:13:40.962818 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:40.962779 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-clngb"] Sep 17 19:13:40.966111 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:40.966086 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-clngb" Sep 17 19:13:40.970499 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:40.970479 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Sep 17 19:13:40.970623 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:40.970567 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Sep 17 19:13:40.970694 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:40.970661 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-nhdv6\"" Sep 17 19:13:40.971448 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:40.971431 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\"" Sep 17 19:13:40.971534 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:40.971500 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Sep 17 19:13:40.985963 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:40.985925 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-clngb"] Sep 17 19:13:41.074863 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:41.074835 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9794b921-7759-4b84-800d-1069b28e4601-model-cache\") pod \"s3-init-clngb\" (UID: \"9794b921-7759-4b84-800d-1069b28e4601\") " pod="kserve/s3-init-clngb" Sep 17 19:13:41.075012 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:41.074880 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97sw9\" (UniqueName: \"kubernetes.io/projected/9794b921-7759-4b84-800d-1069b28e4601-kube-api-access-97sw9\") pod \"s3-init-clngb\" (UID: \"9794b921-7759-4b84-800d-1069b28e4601\") " pod="kserve/s3-init-clngb" Sep 17 19:13:41.176327 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:41.176226 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97sw9\" (UniqueName: \"kubernetes.io/projected/9794b921-7759-4b84-800d-1069b28e4601-kube-api-access-97sw9\") pod \"s3-init-clngb\" (UID: \"9794b921-7759-4b84-800d-1069b28e4601\") " pod="kserve/s3-init-clngb" Sep 17 19:13:41.176491 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:41.176408 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9794b921-7759-4b84-800d-1069b28e4601-model-cache\") pod \"s3-init-clngb\" (UID: \"9794b921-7759-4b84-800d-1069b28e4601\") " pod="kserve/s3-init-clngb" Sep 17 19:13:41.176699 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:41.176686 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9794b921-7759-4b84-800d-1069b28e4601-model-cache\") pod \"s3-init-clngb\" (UID: \"9794b921-7759-4b84-800d-1069b28e4601\") " pod="kserve/s3-init-clngb" Sep 17 19:13:41.188801 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:41.188774 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97sw9\" (UniqueName: \"kubernetes.io/projected/9794b921-7759-4b84-800d-1069b28e4601-kube-api-access-97sw9\") pod \"s3-init-clngb\" (UID: \"9794b921-7759-4b84-800d-1069b28e4601\") " pod="kserve/s3-init-clngb" Sep 17 19:13:41.295364 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:41.295303 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-clngb" Sep 17 19:13:41.569200 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:41.569178 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-clngb"] Sep 17 19:13:41.571776 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:13:41.571747 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9794b921_7759_4b84_800d_1069b28e4601.slice/crio-b452281dbd9e7c150fe0733fbb6c418d0291cc7b59ea38183aff0c625a046b70 WatchSource:0}: Error finding container b452281dbd9e7c150fe0733fbb6c418d0291cc7b59ea38183aff0c625a046b70: Status 404 returned error can't find the container with id b452281dbd9e7c150fe0733fbb6c418d0291cc7b59ea38183aff0c625a046b70 Sep 17 19:13:41.573569 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:41.573554 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 19:13:42.102762 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:42.102724 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clngb" event={"ID":"9794b921-7759-4b84-800d-1069b28e4601","Type":"ContainerStarted","Data":"b452281dbd9e7c150fe0733fbb6c418d0291cc7b59ea38183aff0c625a046b70"} Sep 17 19:13:46.122612 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:46.122569 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clngb" event={"ID":"9794b921-7759-4b84-800d-1069b28e4601","Type":"ContainerStarted","Data":"0d13862b185775ae7b644fc05694050ec50ef13fec52bcf573adb0878284e73c"} Sep 17 19:13:48.133289 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:48.133249 2563 generic.go:356] "Generic (PLEG): container finished" podID="9794b921-7759-4b84-800d-1069b28e4601" containerID="0d13862b185775ae7b644fc05694050ec50ef13fec52bcf573adb0878284e73c" exitCode=0 Sep 17 19:13:48.133699 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:48.133320 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clngb" event={"ID":"9794b921-7759-4b84-800d-1069b28e4601","Type":"ContainerDied","Data":"0d13862b185775ae7b644fc05694050ec50ef13fec52bcf573adb0878284e73c"} Sep 17 19:13:53.158595 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:13:53.158518 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clngb" event={"ID":"9794b921-7759-4b84-800d-1069b28e4601","Type":"ContainerStarted","Data":"b7700b0f14eba38e1ce9c2d1f0b779d534b2c0ba9a6df2591e110b4c778c431e"} Sep 17 19:14:58.422432 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:14:58.422399 2563 generic.go:356] "Generic (PLEG): container finished" podID="9794b921-7759-4b84-800d-1069b28e4601" containerID="b7700b0f14eba38e1ce9c2d1f0b779d534b2c0ba9a6df2591e110b4c778c431e" exitCode=0 Sep 17 19:14:58.422838 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:14:58.422467 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clngb" event={"ID":"9794b921-7759-4b84-800d-1069b28e4601","Type":"ContainerDied","Data":"b7700b0f14eba38e1ce9c2d1f0b779d534b2c0ba9a6df2591e110b4c778c431e"} Sep 17 19:14:59.546754 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:14:59.546730 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-clngb" Sep 17 19:14:59.659880 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:14:59.659846 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9794b921-7759-4b84-800d-1069b28e4601-model-cache\") pod \"9794b921-7759-4b84-800d-1069b28e4601\" (UID: \"9794b921-7759-4b84-800d-1069b28e4601\") " Sep 17 19:14:59.660056 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:14:59.659941 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97sw9\" (UniqueName: \"kubernetes.io/projected/9794b921-7759-4b84-800d-1069b28e4601-kube-api-access-97sw9\") pod \"9794b921-7759-4b84-800d-1069b28e4601\" (UID: \"9794b921-7759-4b84-800d-1069b28e4601\") " Sep 17 19:14:59.662321 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:14:59.662297 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9794b921-7759-4b84-800d-1069b28e4601-kube-api-access-97sw9" (OuterVolumeSpecName: "kube-api-access-97sw9") pod "9794b921-7759-4b84-800d-1069b28e4601" (UID: "9794b921-7759-4b84-800d-1069b28e4601"). InnerVolumeSpecName "kube-api-access-97sw9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:14:59.681585 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:14:59.681510 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9794b921-7759-4b84-800d-1069b28e4601-model-cache" (OuterVolumeSpecName: "model-cache") pod "9794b921-7759-4b84-800d-1069b28e4601" (UID: "9794b921-7759-4b84-800d-1069b28e4601"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 19:14:59.760877 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:14:59.760846 2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9794b921-7759-4b84-800d-1069b28e4601-model-cache\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:14:59.760877 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:14:59.760873 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97sw9\" (UniqueName: \"kubernetes.io/projected/9794b921-7759-4b84-800d-1069b28e4601-kube-api-access-97sw9\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:15:00.438029 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:00.437991 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clngb" event={"ID":"9794b921-7759-4b84-800d-1069b28e4601","Type":"ContainerDied","Data":"b452281dbd9e7c150fe0733fbb6c418d0291cc7b59ea38183aff0c625a046b70"} Sep 17 19:15:00.438029 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:00.438011 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-clngb" Sep 17 19:15:00.438029 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:00.438029 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b452281dbd9e7c150fe0733fbb6c418d0291cc7b59ea38183aff0c625a046b70" Sep 17 19:15:13.125270 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.125230 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf"] Sep 17 19:15:13.125630 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.125539 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9794b921-7759-4b84-800d-1069b28e4601" containerName="download-hf-model" Sep 17 19:15:13.125630 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.125550 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9794b921-7759-4b84-800d-1069b28e4601" containerName="download-hf-model" Sep 17 19:15:13.125630 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.125570 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9794b921-7759-4b84-800d-1069b28e4601" containerName="s3-init" Sep 17 19:15:13.125630 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.125575 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9794b921-7759-4b84-800d-1069b28e4601" containerName="s3-init" Sep 17 19:15:13.125630 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.125628 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9794b921-7759-4b84-800d-1069b28e4601" containerName="s3-init" Sep 17 19:15:13.128530 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.128511 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.131386 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.131355 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Sep 17 19:15:13.131510 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.131355 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Sep 17 19:15:13.131510 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.131443 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 19:15:13.131510 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.131461 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Sep 17 19:15:13.131640 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.131548 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tmb8q\"" Sep 17 19:15:13.132335 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.132324 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Sep 17 19:15:13.132402 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.132354 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Sep 17 19:15:13.146836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.146806 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf"] Sep 17 19:15:13.178147 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.178089 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cab27e33-df01-4804-a20b-6cef569edda9-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.178337 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.178156 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.178337 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.178188 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cab27e33-df01-4804-a20b-6cef569edda9-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.178337 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.178208 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnj7m\" (UniqueName: \"kubernetes.io/projected/cab27e33-df01-4804-a20b-6cef569edda9-kube-api-access-vnj7m\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.178337 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.178274 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.279309 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.279274 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.279309 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.279311 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cab27e33-df01-4804-a20b-6cef569edda9-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.279566 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.279334 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnj7m\" (UniqueName: \"kubernetes.io/projected/cab27e33-df01-4804-a20b-6cef569edda9-kube-api-access-vnj7m\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.279566 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.279377 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.279566 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.279413 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cab27e33-df01-4804-a20b-6cef569edda9-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.279838 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.279819 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cab27e33-df01-4804-a20b-6cef569edda9-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.280197 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.280136 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.280197 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.280144 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.281919 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.281892 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cab27e33-df01-4804-a20b-6cef569edda9-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.293035 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.293006 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnj7m\" (UniqueName: \"kubernetes.io/projected/cab27e33-df01-4804-a20b-6cef569edda9-kube-api-access-vnj7m\") pod \"isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.439392 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.439299 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:13.628798 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.628762 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf"] Sep 17 19:15:13.631722 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.631703 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.634341 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.634323 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Sep 17 19:15:13.634424 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.634345 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Sep 17 19:15:13.643314 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.643290 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf"] Sep 17 19:15:13.683066 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.683035 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.683236 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.683071 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.683236 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.683189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a4a74122-c9f0-4285-b7a8-f9a185edfd53-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.683236 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.683228 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb68v\" (UniqueName: \"kubernetes.io/projected/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-api-access-vb68v\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.683333 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.683281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.716652 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.716625 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf"] Sep 17 19:15:13.719249 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:15:13.719220 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcab27e33_df01_4804_a20b_6cef569edda9.slice/crio-cee7cbdecafbc4ca9273a2a4179d65996653735a8e933e5b69dc43dad84cfaab WatchSource:0}: Error finding container cee7cbdecafbc4ca9273a2a4179d65996653735a8e933e5b69dc43dad84cfaab: Status 404 returned error can't find the container with id cee7cbdecafbc4ca9273a2a4179d65996653735a8e933e5b69dc43dad84cfaab Sep 17 19:15:13.784172 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.784140 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.784309 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.784199 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.784309 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.784258 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a4a74122-c9f0-4285-b7a8-f9a185edfd53-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.784409 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.784364 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb68v\" (UniqueName: \"kubernetes.io/projected/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-api-access-vb68v\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.784409 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:15:13.784382 2563 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-predictor-serving-cert: secret "isvc-xgboost-graph-predictor-serving-cert" not found Sep 17 19:15:13.784491 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:15:13.784440 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4a74122-c9f0-4285-b7a8-f9a185edfd53-proxy-tls podName:a4a74122-c9f0-4285-b7a8-f9a185edfd53 nodeName:}" failed. No retries permitted until 2026-09-17 19:15:14.28441989 +0000 UTC m=+522.546293282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/a4a74122-c9f0-4285-b7a8-f9a185edfd53-proxy-tls") pod "isvc-xgboost-graph-predictor-555fc69d56-nsgkf" (UID: "a4a74122-c9f0-4285-b7a8-f9a185edfd53") : secret "isvc-xgboost-graph-predictor-serving-cert" not found Sep 17 19:15:13.784491 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.784471 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.784608 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.784536 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.784904 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.784883 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.785192 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.785172 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:13.793869 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:13.793846 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb68v\" (UniqueName: \"kubernetes.io/projected/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-api-access-vb68v\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:14.289334 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:14.289296 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a4a74122-c9f0-4285-b7a8-f9a185edfd53-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:14.291774 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:14.291744 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a4a74122-c9f0-4285-b7a8-f9a185edfd53-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-nsgkf\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:14.493346 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:14.493311 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" event={"ID":"cab27e33-df01-4804-a20b-6cef569edda9","Type":"ContainerStarted","Data":"4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a"} Sep 17 19:15:14.493527 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:14.493366 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" event={"ID":"cab27e33-df01-4804-a20b-6cef569edda9","Type":"ContainerStarted","Data":"cee7cbdecafbc4ca9273a2a4179d65996653735a8e933e5b69dc43dad84cfaab"} Sep 17 19:15:14.543148 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:14.543038 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:14.831094 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:14.830903 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf"] Sep 17 19:15:14.833707 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:15:14.833677 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a74122_c9f0_4285_b7a8_f9a185edfd53.slice/crio-c6fed058f0adcea5e757fd1670e5de40201f191ab9375a265f2bac507747d835 WatchSource:0}: Error finding container c6fed058f0adcea5e757fd1670e5de40201f191ab9375a265f2bac507747d835: Status 404 returned error can't find the container with id c6fed058f0adcea5e757fd1670e5de40201f191ab9375a265f2bac507747d835 Sep 17 19:15:15.506305 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:15.506269 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" event={"ID":"a4a74122-c9f0-4285-b7a8-f9a185edfd53","Type":"ContainerStarted","Data":"28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010"} Sep 17 19:15:15.506727 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:15.506350 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" event={"ID":"a4a74122-c9f0-4285-b7a8-f9a185edfd53","Type":"ContainerStarted","Data":"c6fed058f0adcea5e757fd1670e5de40201f191ab9375a265f2bac507747d835"} Sep 17 19:15:18.522600 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:18.522569 2563 generic.go:356] "Generic (PLEG): container finished" podID="cab27e33-df01-4804-a20b-6cef569edda9" containerID="4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a" exitCode=0 Sep 17 19:15:18.522965 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:18.522661 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" event={"ID":"cab27e33-df01-4804-a20b-6cef569edda9","Type":"ContainerDied","Data":"4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a"} Sep 17 19:15:19.531605 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:19.531566 2563 generic.go:356] "Generic (PLEG): container finished" podID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerID="28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010" exitCode=0 Sep 17 19:15:19.532055 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:19.531713 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" event={"ID":"a4a74122-c9f0-4285-b7a8-f9a185edfd53","Type":"ContainerDied","Data":"28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010"} Sep 17 19:15:43.667784 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:43.667750 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" event={"ID":"cab27e33-df01-4804-a20b-6cef569edda9","Type":"ContainerStarted","Data":"12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7"} Sep 17 19:15:44.675914 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:44.675875 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" event={"ID":"a4a74122-c9f0-4285-b7a8-f9a185edfd53","Type":"ContainerStarted","Data":"9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6"} Sep 17 19:15:46.686714 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:46.686678 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" event={"ID":"a4a74122-c9f0-4285-b7a8-f9a185edfd53","Type":"ContainerStarted","Data":"f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7"} Sep 17 19:15:46.687183 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:46.686761 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:46.688910 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:46.688887 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" event={"ID":"cab27e33-df01-4804-a20b-6cef569edda9","Type":"ContainerStarted","Data":"b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7"} Sep 17 19:15:46.689026 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:46.688933 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:46.689026 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:46.689000 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:46.690319 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:46.690297 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Sep 17 19:15:46.718140 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:46.718067 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podStartSLOduration=3.642090758 podStartE2EDuration="33.718050419s" totalImagesPullingTime="26.282230544s" totalInitContainerRuntime="3.793729117s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:15:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:15:46.717853663 +0000 UTC m=+554.979727079" watchObservedRunningTime="2026-09-17 19:15:46.718050419 +0000 UTC m=+554.979923824" Sep 17 19:15:46.741457 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:46.741401 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podStartSLOduration=2.514299727 podStartE2EDuration="33.741378407s" totalImagesPullingTime="27.284765708s" totalInitContainerRuntime="3.942312972s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:15:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:15:46.741122544 +0000 UTC m=+555.002995961" watchObservedRunningTime="2026-09-17 19:15:46.741378407 +0000 UTC m=+555.003251815" Sep 17 19:15:47.693433 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:47.693396 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Sep 17 19:15:47.693846 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:47.693476 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:47.694669 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:47.694630 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 19:15:48.697434 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:48.697396 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 19:15:52.697782 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:52.697756 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:15:52.698308 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:52.698279 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Sep 17 19:15:53.702165 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:53.702132 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:15:53.702692 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:15:53.702657 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 19:16:02.698626 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:02.698572 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Sep 17 19:16:03.703144 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:03.703077 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 19:16:12.698242 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:12.698193 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Sep 17 19:16:13.702794 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:13.702742 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 19:16:22.698236 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:22.698192 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Sep 17 19:16:23.702854 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:23.702808 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 19:16:32.136659 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:32.136631 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:16:32.140148 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:32.140126 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:16:32.698213 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:32.698179 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Sep 17 19:16:33.703645 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:33.703599 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 19:16:42.698260 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:42.698219 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Sep 17 19:16:43.703175 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:43.703127 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 19:16:52.698353 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:52.698311 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Sep 17 19:16:53.703597 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:53.703510 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:16:54.229836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:16:54.229794 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused" Sep 17 19:17:04.234227 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:04.234192 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:17:44.699953 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:44.699917 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf"] Sep 17 19:17:44.701498 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:44.700266 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" containerID="cri-o://12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7" gracePeriod=30 Sep 17 19:17:44.701498 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:44.700291 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kube-rbac-proxy" containerID="cri-o://b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7" gracePeriod=30 Sep 17 19:17:44.751016 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:44.750972 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf"] Sep 17 19:17:44.751332 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:44.751292 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" containerID="cri-o://9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6" gracePeriod=30 Sep 17 19:17:44.751418 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:44.751371 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kube-rbac-proxy" containerID="cri-o://f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7" gracePeriod=30 Sep 17 19:17:45.179925 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:45.179891 2563 generic.go:356] "Generic (PLEG): container finished" podID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerID="f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7" exitCode=2 Sep 17 19:17:45.180131 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:45.179964 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" event={"ID":"a4a74122-c9f0-4285-b7a8-f9a185edfd53","Type":"ContainerDied","Data":"f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7"} Sep 17 19:17:45.182166 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:45.182140 2563 generic.go:356] "Generic (PLEG): container finished" podID="cab27e33-df01-4804-a20b-6cef569edda9" containerID="b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7" exitCode=2 Sep 17 19:17:45.182264 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:45.182176 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" event={"ID":"cab27e33-df01-4804-a20b-6cef569edda9","Type":"ContainerDied","Data":"b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7"} Sep 17 19:17:47.693582 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:47.693543 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.24:8643/healthz\": dial tcp 10.132.0.24:8643: connect: connection refused" Sep 17 19:17:48.698405 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:48.698340 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.25:8643/healthz\": dial tcp 10.132.0.25:8643: connect: connection refused" Sep 17 19:17:49.804411 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:49.804383 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:17:49.911969 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:49.911877 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kserve-provision-location\") pod \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " Sep 17 19:17:49.911969 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:49.911925 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vb68v\" (UniqueName: \"kubernetes.io/projected/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-api-access-vb68v\") pod \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " Sep 17 19:17:49.912252 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:49.911994 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a4a74122-c9f0-4285-b7a8-f9a185edfd53-proxy-tls\") pod \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " Sep 17 19:17:49.912252 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:49.912028 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-cabundle-cert\") pod \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " Sep 17 19:17:49.912252 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:49.912066 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-rbac-proxy-sar-config\") pod \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\" (UID: \"a4a74122-c9f0-4285-b7a8-f9a185edfd53\") " Sep 17 19:17:49.912401 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:49.912297 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "a4a74122-c9f0-4285-b7a8-f9a185edfd53" (UID: "a4a74122-c9f0-4285-b7a8-f9a185edfd53"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 19:17:49.912493 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:49.912464 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "a4a74122-c9f0-4285-b7a8-f9a185edfd53" (UID: "a4a74122-c9f0-4285-b7a8-f9a185edfd53"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:17:49.912532 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:49.912492 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a4a74122-c9f0-4285-b7a8-f9a185edfd53" (UID: "a4a74122-c9f0-4285-b7a8-f9a185edfd53"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:17:49.914505 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:49.914482 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-api-access-vb68v" (OuterVolumeSpecName: "kube-api-access-vb68v") pod "a4a74122-c9f0-4285-b7a8-f9a185edfd53" (UID: "a4a74122-c9f0-4285-b7a8-f9a185edfd53"). InnerVolumeSpecName "kube-api-access-vb68v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:17:49.914606 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:49.914505 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a74122-c9f0-4285-b7a8-f9a185edfd53-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a4a74122-c9f0-4285-b7a8-f9a185edfd53" (UID: "a4a74122-c9f0-4285-b7a8-f9a185edfd53"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:17:50.013322 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.013287 2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a4a74122-c9f0-4285-b7a8-f9a185edfd53-proxy-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.013322 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.013316 2563 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-cabundle-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.013322 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.013327 2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.013551 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.013337 2563 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kserve-provision-location\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.013551 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.013346 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb68v\" (UniqueName: \"kubernetes.io/projected/a4a74122-c9f0-4285-b7a8-f9a185edfd53-kube-api-access-vb68v\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.206785 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.206748 2563 generic.go:356] "Generic (PLEG): container finished" podID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerID="9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6" exitCode=0 Sep 17 19:17:50.206935 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.206811 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" Sep 17 19:17:50.206935 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.206835 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" event={"ID":"a4a74122-c9f0-4285-b7a8-f9a185edfd53","Type":"ContainerDied","Data":"9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6"} Sep 17 19:17:50.206935 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.206907 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf" event={"ID":"a4a74122-c9f0-4285-b7a8-f9a185edfd53","Type":"ContainerDied","Data":"c6fed058f0adcea5e757fd1670e5de40201f191ab9375a265f2bac507747d835"} Sep 17 19:17:50.207039 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.206936 2563 scope.go:117] "RemoveContainer" containerID="f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7" Sep 17 19:17:50.216131 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.216084 2563 scope.go:117] "RemoveContainer" containerID="9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6" Sep 17 19:17:50.224790 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.224769 2563 scope.go:117] "RemoveContainer" containerID="28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010" Sep 17 19:17:50.234807 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.234785 2563 scope.go:117] "RemoveContainer" containerID="f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7" Sep 17 19:17:50.235145 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:17:50.235120 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7\": container with ID starting with f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7 not found: ID does not exist" containerID="f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7" Sep 17 19:17:50.235212 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.235152 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7"} err="failed to get container status \"f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7\": rpc error: code = NotFound desc = could not find container \"f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7\": container with ID starting with f8aad4a3e9c869eb461b4a39a71d778d9ace1b93b33010f258a80faad5b025e7 not found: ID does not exist" Sep 17 19:17:50.235212 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.235170 2563 scope.go:117] "RemoveContainer" containerID="9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6" Sep 17 19:17:50.235465 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:17:50.235447 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6\": container with ID starting with 9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6 not found: ID does not exist" containerID="9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6" Sep 17 19:17:50.235509 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.235473 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6"} err="failed to get container status \"9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6\": rpc error: code = NotFound desc = could not find container \"9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6\": container with ID starting with 9929a5ad695e87f93fe4afaef91976fb8537f40ba94ba34af944908b9400a6a6 not found: ID does not exist" Sep 17 19:17:50.235509 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.235491 2563 scope.go:117] "RemoveContainer" containerID="28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010" Sep 17 19:17:50.235578 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.235563 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf"] Sep 17 19:17:50.235615 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.235590 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-nsgkf"] Sep 17 19:17:50.235786 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:17:50.235770 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010\": container with ID starting with 28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010 not found: ID does not exist" containerID="28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010" Sep 17 19:17:50.235826 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.235791 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010"} err="failed to get container status \"28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010\": rpc error: code = NotFound desc = could not find container \"28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010\": container with ID starting with 28774dc20cf7436d3c5f038fb8d6712d717b479d0ccec66a4d75ee116a5c1010 not found: ID does not exist" Sep 17 19:17:50.746421 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.746392 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:17:50.819528 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.819436 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-cabundle-cert\") pod \"cab27e33-df01-4804-a20b-6cef569edda9\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " Sep 17 19:17:50.819947 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.819544 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-kube-rbac-proxy-sar-config\") pod \"cab27e33-df01-4804-a20b-6cef569edda9\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " Sep 17 19:17:50.819947 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.819569 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cab27e33-df01-4804-a20b-6cef569edda9-proxy-tls\") pod \"cab27e33-df01-4804-a20b-6cef569edda9\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " Sep 17 19:17:50.819947 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.819602 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cab27e33-df01-4804-a20b-6cef569edda9-kserve-provision-location\") pod \"cab27e33-df01-4804-a20b-6cef569edda9\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " Sep 17 19:17:50.819947 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.819628 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnj7m\" (UniqueName: \"kubernetes.io/projected/cab27e33-df01-4804-a20b-6cef569edda9-kube-api-access-vnj7m\") pod \"cab27e33-df01-4804-a20b-6cef569edda9\" (UID: \"cab27e33-df01-4804-a20b-6cef569edda9\") " Sep 17 19:17:50.819947 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.819922 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cab27e33-df01-4804-a20b-6cef569edda9-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "cab27e33-df01-4804-a20b-6cef569edda9" (UID: "cab27e33-df01-4804-a20b-6cef569edda9"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 19:17:50.819947 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.819936 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "cab27e33-df01-4804-a20b-6cef569edda9" (UID: "cab27e33-df01-4804-a20b-6cef569edda9"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:17:50.820192 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.819948 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "cab27e33-df01-4804-a20b-6cef569edda9" (UID: "cab27e33-df01-4804-a20b-6cef569edda9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:17:50.821946 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.821922 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cab27e33-df01-4804-a20b-6cef569edda9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cab27e33-df01-4804-a20b-6cef569edda9" (UID: "cab27e33-df01-4804-a20b-6cef569edda9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:17:50.821946 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.821931 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cab27e33-df01-4804-a20b-6cef569edda9-kube-api-access-vnj7m" (OuterVolumeSpecName: "kube-api-access-vnj7m") pod "cab27e33-df01-4804-a20b-6cef569edda9" (UID: "cab27e33-df01-4804-a20b-6cef569edda9"). InnerVolumeSpecName "kube-api-access-vnj7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:17:50.920356 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.920322 2563 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.920356 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.920350 2563 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cab27e33-df01-4804-a20b-6cef569edda9-proxy-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.920356 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.920362 2563 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cab27e33-df01-4804-a20b-6cef569edda9-kserve-provision-location\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.920588 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.920370 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnj7m\" (UniqueName: \"kubernetes.io/projected/cab27e33-df01-4804-a20b-6cef569edda9-kube-api-access-vnj7m\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.920588 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:50.920381 2563 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cab27e33-df01-4804-a20b-6cef569edda9-cabundle-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:17:51.214228 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.214194 2563 generic.go:356] "Generic (PLEG): container finished" podID="cab27e33-df01-4804-a20b-6cef569edda9" containerID="12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7" exitCode=0 Sep 17 19:17:51.214412 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.214242 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" event={"ID":"cab27e33-df01-4804-a20b-6cef569edda9","Type":"ContainerDied","Data":"12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7"} Sep 17 19:17:51.214412 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.214254 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" Sep 17 19:17:51.214412 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.214290 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf" event={"ID":"cab27e33-df01-4804-a20b-6cef569edda9","Type":"ContainerDied","Data":"cee7cbdecafbc4ca9273a2a4179d65996653735a8e933e5b69dc43dad84cfaab"} Sep 17 19:17:51.214412 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.214317 2563 scope.go:117] "RemoveContainer" containerID="b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7" Sep 17 19:17:51.222919 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.222901 2563 scope.go:117] "RemoveContainer" containerID="12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7" Sep 17 19:17:51.231308 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.231290 2563 scope.go:117] "RemoveContainer" containerID="4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a" Sep 17 19:17:51.238058 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.238028 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf"] Sep 17 19:17:51.239677 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.239658 2563 scope.go:117] "RemoveContainer" containerID="b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7" Sep 17 19:17:51.240321 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:17:51.240080 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7\": container with ID starting with b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7 not found: ID does not exist" containerID="b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7" Sep 17 19:17:51.240321 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.240138 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7"} err="failed to get container status \"b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7\": rpc error: code = NotFound desc = could not find container \"b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7\": container with ID starting with b90f63d536cce3aa6db7a1c390b86b302292ec6424e343ac58262d0d9e9319d7 not found: ID does not exist" Sep 17 19:17:51.240321 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.240164 2563 scope.go:117] "RemoveContainer" containerID="12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7" Sep 17 19:17:51.240835 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:17:51.240537 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7\": container with ID starting with 12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7 not found: ID does not exist" containerID="12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7" Sep 17 19:17:51.240835 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.240574 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7"} err="failed to get container status \"12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7\": rpc error: code = NotFound desc = could not find container \"12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7\": container with ID starting with 12f4adaaedede5e9e9073a646e10c93dbb81f22db805bc3e3063c89c5b8f71b7 not found: ID does not exist" Sep 17 19:17:51.240835 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.240599 2563 scope.go:117] "RemoveContainer" containerID="4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a" Sep 17 19:17:51.240994 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:17:51.240920 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a\": container with ID starting with 4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a not found: ID does not exist" containerID="4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a" Sep 17 19:17:51.240994 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.240942 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a"} err="failed to get container status \"4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a\": rpc error: code = NotFound desc = could not find container \"4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a\": container with ID starting with 4f19701013735a6b404a0d8de4a9140a38f7b088a537c79d6a42f7833ed7315a not found: ID does not exist" Sep 17 19:17:51.242230 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:51.242208 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-64d8fbb9c6-xx5pf"] Sep 17 19:17:52.240295 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:52.240257 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" path="/var/lib/kubelet/pods/a4a74122-c9f0-4285-b7a8-f9a185edfd53/volumes" Sep 17 19:17:52.240761 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:17:52.240747 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cab27e33-df01-4804-a20b-6cef569edda9" path="/var/lib/kubelet/pods/cab27e33-df01-4804-a20b-6cef569edda9/volumes" Sep 17 19:21:32.162995 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:21:32.162925 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:21:32.164209 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:21:32.164190 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:26:32.191346 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:26:32.191316 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:26:32.191795 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:26:32.191604 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:31:32.217454 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:31:32.217430 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:31:32.217966 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:31:32.217697 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:36:32.244667 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:36:32.244637 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:36:32.245083 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:36:32.244821 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:41:32.268695 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:41:32.268660 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:41:32.271038 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:41:32.270509 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:46:32.293892 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:46:32.293789 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:46:32.297952 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:46:32.296400 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:51:32.322317 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:51:32.322211 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:51:32.326233 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:51:32.325478 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:55:28.310358 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310322 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hr64j/must-gather-bbxrr"] Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310677 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310693 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310702 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="storage-initializer" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310708 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="storage-initializer" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310722 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="storage-initializer" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310728 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="storage-initializer" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310735 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310740 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310751 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kube-rbac-proxy" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310755 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kube-rbac-proxy" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310763 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kube-rbac-proxy" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310768 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kube-rbac-proxy" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310810 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kserve-container" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310819 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kserve-container" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310825 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cab27e33-df01-4804-a20b-6cef569edda9" containerName="kube-rbac-proxy" Sep 17 19:55:28.310836 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.310832 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a74122-c9f0-4285-b7a8-f9a185edfd53" containerName="kube-rbac-proxy" Sep 17 19:55:28.313805 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.313790 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:55:28.316402 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.316374 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hr64j\"/\"default-dockercfg-vxlzl\"" Sep 17 19:55:28.316402 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.316394 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hr64j\"/\"kube-root-ca.crt\"" Sep 17 19:55:28.316578 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.316374 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hr64j\"/\"openshift-service-ca.crt\"" Sep 17 19:55:28.323086 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.323059 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hr64j/must-gather-bbxrr"] Sep 17 19:55:28.487932 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.487902 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-must-gather-output\") pod \"must-gather-bbxrr\" (UID: \"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94\") " pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:55:28.488094 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.487940 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxpm4\" (UniqueName: \"kubernetes.io/projected/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-kube-api-access-kxpm4\") pod \"must-gather-bbxrr\" (UID: \"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94\") " pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:55:28.588567 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.588494 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-must-gather-output\") pod \"must-gather-bbxrr\" (UID: \"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94\") " pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:55:28.588567 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.588540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxpm4\" (UniqueName: \"kubernetes.io/projected/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-kube-api-access-kxpm4\") pod \"must-gather-bbxrr\" (UID: \"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94\") " pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:55:28.588802 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.588784 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-must-gather-output\") pod \"must-gather-bbxrr\" (UID: \"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94\") " pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:55:28.596963 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.596936 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxpm4\" (UniqueName: \"kubernetes.io/projected/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-kube-api-access-kxpm4\") pod \"must-gather-bbxrr\" (UID: \"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94\") " pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:55:28.633498 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.633480 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:55:28.899642 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.899617 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hr64j/must-gather-bbxrr"] Sep 17 19:55:28.901921 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:55:28.901888 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24c50e23_e2f2_4efe_b0e4_ca62b4d80e94.slice/crio-4dea9e8ae83dc11f4b85303ae43ff33dacbed4cf09d8226cc09fbf3566dd2f58 WatchSource:0}: Error finding container 4dea9e8ae83dc11f4b85303ae43ff33dacbed4cf09d8226cc09fbf3566dd2f58: Status 404 returned error can't find the container with id 4dea9e8ae83dc11f4b85303ae43ff33dacbed4cf09d8226cc09fbf3566dd2f58 Sep 17 19:55:28.903628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:28.903613 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 19:55:29.891290 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:29.891228 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hr64j/must-gather-bbxrr" event={"ID":"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94","Type":"ContainerStarted","Data":"4dea9e8ae83dc11f4b85303ae43ff33dacbed4cf09d8226cc09fbf3566dd2f58"} Sep 17 19:55:35.923323 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:35.923285 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hr64j/must-gather-bbxrr" event={"ID":"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94","Type":"ContainerStarted","Data":"b318089d000edee484379af8691ddae419da36c234d392b5302b1d72aaf67ffb"} Sep 17 19:55:35.923729 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:35.923337 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hr64j/must-gather-bbxrr" event={"ID":"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94","Type":"ContainerStarted","Data":"8f98e668f75ba7ecf67fed3f532402ada71b23a29c341874595a612bd8edb2dd"} Sep 17 19:55:35.938942 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:35.938893 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hr64j/must-gather-bbxrr" podStartSLOduration=1.7047132550000001 podStartE2EDuration="7.938876933s" totalImagesPullingTime="6.234163678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:55:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:55:35.937602441 +0000 UTC m=+2944.199475847" watchObservedRunningTime="2026-09-17 19:55:35.938876933 +0000 UTC m=+2944.200750338" Sep 17 19:55:59.020571 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:59.020535 2563 generic.go:356] "Generic (PLEG): container finished" podID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" containerID="8f98e668f75ba7ecf67fed3f532402ada71b23a29c341874595a612bd8edb2dd" exitCode=0 Sep 17 19:55:59.020985 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:59.020607 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hr64j/must-gather-bbxrr" event={"ID":"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94","Type":"ContainerDied","Data":"8f98e668f75ba7ecf67fed3f532402ada71b23a29c341874595a612bd8edb2dd"} Sep 17 19:55:59.020985 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:59.020903 2563 scope.go:117] "RemoveContainer" containerID="8f98e668f75ba7ecf67fed3f532402ada71b23a29c341874595a612bd8edb2dd" Sep 17 19:55:59.419677 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:55:59.419611 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-hr64j_must-gather-bbxrr_24c50e23-e2f2-4efe-b0e4-ca62b4d80e94/gather/0.log" Sep 17 19:56:03.081975 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:03.081951 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xvjpg_7d66c218-c6ed-4ca6-a49a-47bdc94fef2d/global-pull-secret-syncer/0.log" Sep 17 19:56:03.164686 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:03.164665 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-94j6j_58883cfb-501e-4277-9ece-f19ed6452fb7/konnectivity-agent/0.log" Sep 17 19:56:03.244968 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:03.244938 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-16.ec2.internal_aac0a2a35937da63c2df9c101e366883/haproxy/0.log" Sep 17 19:56:04.862372 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:04.862337 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-hr64j/must-gather-bbxrr"] Sep 17 19:56:04.862911 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:04.862538 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-hr64j/must-gather-bbxrr" podUID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" containerName="copy" containerID="cri-o://b318089d000edee484379af8691ddae419da36c234d392b5302b1d72aaf67ffb" gracePeriod=2 Sep 17 19:56:04.867396 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:04.867370 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-hr64j/must-gather-bbxrr"] Sep 17 19:56:05.047808 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:05.047787 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-hr64j_must-gather-bbxrr_24c50e23-e2f2-4efe-b0e4-ca62b4d80e94/copy/0.log" Sep 17 19:56:05.048182 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:05.048159 2563 generic.go:356] "Generic (PLEG): container finished" podID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" containerID="b318089d000edee484379af8691ddae419da36c234d392b5302b1d72aaf67ffb" exitCode=143 Sep 17 19:56:05.098580 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:05.098560 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-hr64j_must-gather-bbxrr_24c50e23-e2f2-4efe-b0e4-ca62b4d80e94/copy/0.log" Sep 17 19:56:05.100283 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:05.100266 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:56:05.102251 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:56:05.102225 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-bbxrr\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-hr64j\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:56:05.196634 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:05.196577 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-must-gather-output\") pod \"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94\" (UID: \"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94\") " Sep 17 19:56:05.196634 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:05.196628 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxpm4\" (UniqueName: \"kubernetes.io/projected/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-kube-api-access-kxpm4\") pod \"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94\" (UID: \"24c50e23-e2f2-4efe-b0e4-ca62b4d80e94\") " Sep 17 19:56:05.198119 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:05.198081 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" (UID: "24c50e23-e2f2-4efe-b0e4-ca62b4d80e94"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 19:56:05.198928 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:05.198897 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-kube-api-access-kxpm4" (OuterVolumeSpecName: "kube-api-access-kxpm4") pod "24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" (UID: "24c50e23-e2f2-4efe-b0e4-ca62b4d80e94"). InnerVolumeSpecName "kube-api-access-kxpm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:56:05.297561 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:05.297525 2563 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-must-gather-output\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:56:05.297561 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:05.297556 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxpm4\" (UniqueName: \"kubernetes.io/projected/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94-kube-api-access-kxpm4\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\"" Sep 17 19:56:06.053546 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:06.053519 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-hr64j_must-gather-bbxrr_24c50e23-e2f2-4efe-b0e4-ca62b4d80e94/copy/0.log" Sep 17 19:56:06.053975 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:06.053918 2563 scope.go:117] "RemoveContainer" containerID="b318089d000edee484379af8691ddae419da36c234d392b5302b1d72aaf67ffb" Sep 17 19:56:06.053975 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:06.053951 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:56:06.056117 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:56:06.056066 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-bbxrr\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-hr64j\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:56:06.061984 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:06.061969 2563 scope.go:117] "RemoveContainer" containerID="8f98e668f75ba7ecf67fed3f532402ada71b23a29c341874595a612bd8edb2dd" Sep 17 19:56:06.063589 ip-10-0-135-16 kubenswrapper[2563]: E0917 19:56:06.063562 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-bbxrr\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-hr64j\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" pod="openshift-must-gather-hr64j/must-gather-bbxrr" Sep 17 19:56:06.234660 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:06.234622 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" path="/var/lib/kubelet/pods/24c50e23-e2f2-4efe-b0e4-ca62b4d80e94/volumes" Sep 17 19:56:07.342324 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:07.342295 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-jb6mm_8df8584c-c2db-416a-a790-6a51db7aea0e/cluster-monitoring-operator/0.log" Sep 17 19:56:07.374450 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:07.374427 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-qfpj5_5842211a-e5f4-46a5-abdc-765ec50dcd98/kube-state-metrics/0.log" Sep 17 19:56:07.408385 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:07.408368 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-qfpj5_5842211a-e5f4-46a5-abdc-765ec50dcd98/kube-rbac-proxy-main/0.log" Sep 17 19:56:07.436012 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:07.435995 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-qfpj5_5842211a-e5f4-46a5-abdc-765ec50dcd98/kube-rbac-proxy-self/0.log" Sep 17 19:56:07.465996 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:07.465976 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-664fc8f54d-fclsg_2e737871-bcb4-4d88-bd9e-99176999cedf/metrics-server/0.log" Sep 17 19:56:07.749067 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:07.749008 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zrzgc_7e16b1bb-d157-4f46-85c4-6ce3e028886d/node-exporter/0.log" Sep 17 19:56:07.774775 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:07.774753 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zrzgc_7e16b1bb-d157-4f46-85c4-6ce3e028886d/kube-rbac-proxy/0.log" Sep 17 19:56:07.800473 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:07.800455 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zrzgc_7e16b1bb-d157-4f46-85c4-6ce3e028886d/init-textfile/0.log" Sep 17 19:56:07.935348 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:07.935329 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_560326e5-d224-4110-9b53-e8bbaf2ddace/prometheus/0.log" Sep 17 19:56:07.957453 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:07.957435 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_560326e5-d224-4110-9b53-e8bbaf2ddace/config-reloader/0.log" Sep 17 19:56:07.988275 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:07.988257 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_560326e5-d224-4110-9b53-e8bbaf2ddace/thanos-sidecar/0.log" Sep 17 19:56:08.013084 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.013032 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_560326e5-d224-4110-9b53-e8bbaf2ddace/kube-rbac-proxy-web/0.log" Sep 17 19:56:08.038782 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.038767 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_560326e5-d224-4110-9b53-e8bbaf2ddace/kube-rbac-proxy/0.log" Sep 17 19:56:08.070567 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.070549 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_560326e5-d224-4110-9b53-e8bbaf2ddace/kube-rbac-proxy-thanos/0.log" Sep 17 19:56:08.094966 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.094947 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_560326e5-d224-4110-9b53-e8bbaf2ddace/init-config-reloader/0.log" Sep 17 19:56:08.126255 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.126239 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-lpfwq_c3a605cb-4839-472d-9890-3fe41ae3dd44/prometheus-operator/0.log" Sep 17 19:56:08.151563 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.151549 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-lpfwq_c3a605cb-4839-472d-9890-3fe41ae3dd44/kube-rbac-proxy/0.log" Sep 17 19:56:08.332875 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.332807 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d99bc4c6f-grdmp_457a7e56-d003-435f-b974-a2b77056babb/thanos-query/0.log" Sep 17 19:56:08.358371 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.358339 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d99bc4c6f-grdmp_457a7e56-d003-435f-b974-a2b77056babb/kube-rbac-proxy-web/0.log" Sep 17 19:56:08.382399 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.382375 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d99bc4c6f-grdmp_457a7e56-d003-435f-b974-a2b77056babb/kube-rbac-proxy/0.log" Sep 17 19:56:08.405643 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.405624 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d99bc4c6f-grdmp_457a7e56-d003-435f-b974-a2b77056babb/prom-label-proxy/0.log" Sep 17 19:56:08.430243 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.430223 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d99bc4c6f-grdmp_457a7e56-d003-435f-b974-a2b77056babb/kube-rbac-proxy-rules/0.log" Sep 17 19:56:08.458583 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:08.458561 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d99bc4c6f-grdmp_457a7e56-d003-435f-b974-a2b77056babb/kube-rbac-proxy-metrics/0.log" Sep 17 19:56:10.361291 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:10.361260 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/1.log" Sep 17 19:56:10.366748 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:10.366728 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-5c69q_e0a5aab2-9dee-4f53-b571-4c0be55c77e0/console-operator/2.log" Sep 17 19:56:11.419423 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.419393 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-c8pst_3e4c6b0f-837d-4eca-b8f4-c922e1de910d/volume-data-source-validator/0.log" Sep 17 19:56:11.761302 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.761271 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb"] Sep 17 19:56:11.761592 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.761579 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" containerName="gather" Sep 17 19:56:11.761642 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.761594 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" containerName="gather" Sep 17 19:56:11.761642 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.761615 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" containerName="copy" Sep 17 19:56:11.761642 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.761620 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" containerName="copy" Sep 17 19:56:11.761733 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.761674 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" containerName="gather" Sep 17 19:56:11.761733 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.761682 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c50e23-e2f2-4efe-b0e4-ca62b4d80e94" containerName="copy" Sep 17 19:56:11.763753 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.763733 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.765997 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.765981 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9dgbs\"/\"kube-root-ca.crt\"" Sep 17 19:56:11.766839 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.766817 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9dgbs\"/\"default-dockercfg-lfrpr\"" Sep 17 19:56:11.766964 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.766841 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9dgbs\"/\"openshift-service-ca.crt\"" Sep 17 19:56:11.772775 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.772755 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb"] Sep 17 19:56:11.847519 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.847494 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-lib-modules\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.847628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.847530 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-sys\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.847628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.847553 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-proc\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.847628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.847570 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpcv8\" (UniqueName: \"kubernetes.io/projected/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-kube-api-access-vpcv8\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.847628 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.847594 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-podres\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.948243 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.948212 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-sys\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.948380 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.948251 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-proc\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.948380 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.948273 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpcv8\" (UniqueName: \"kubernetes.io/projected/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-kube-api-access-vpcv8\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.948380 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.948296 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-podres\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.948380 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.948338 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-sys\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.948380 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.948347 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-proc\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.948380 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.948357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-lib-modules\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.948624 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.948422 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-podres\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.948624 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.948510 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-lib-modules\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:11.956550 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:11.956522 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpcv8\" (UniqueName: \"kubernetes.io/projected/1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f-kube-api-access-vpcv8\") pod \"perf-node-gather-daemonset-7v2cb\" (UID: \"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:12.074118 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:12.074037 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:12.341859 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:12.341792 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb"] Sep 17 19:56:12.344685 ip-10-0-135-16 kubenswrapper[2563]: W0917 19:56:12.344640 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1d221cbf_21fb_4ba9_b976_6cdd6bf01e4f.slice/crio-e5bc7a9760ec05df5df070e5d8b56b8703e453e75deaa6b8cacf8feb3199f385 WatchSource:0}: Error finding container e5bc7a9760ec05df5df070e5d8b56b8703e453e75deaa6b8cacf8feb3199f385: Status 404 returned error can't find the container with id e5bc7a9760ec05df5df070e5d8b56b8703e453e75deaa6b8cacf8feb3199f385 Sep 17 19:56:12.511862 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:12.511841 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fbrxt_92e91b8e-d907-451f-8653-29bd65f28caa/dns/0.log" Sep 17 19:56:12.536737 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:12.536718 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fbrxt_92e91b8e-d907-451f-8653-29bd65f28caa/kube-rbac-proxy/0.log" Sep 17 19:56:12.588265 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:12.588243 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l6h2d_ac28db00-3290-4731-ab0c-9963ef1d6560/dns-node-resolver/0.log" Sep 17 19:56:13.085543 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:13.085501 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" event={"ID":"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f","Type":"ContainerStarted","Data":"70c2b574a7d121719092de925abd0cd95cd426fbe328febf6240883333c1bda6"} Sep 17 19:56:13.085721 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:13.085596 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" event={"ID":"1d221cbf-21fb-4ba9-b976-6cdd6bf01e4f","Type":"ContainerStarted","Data":"e5bc7a9760ec05df5df070e5d8b56b8703e453e75deaa6b8cacf8feb3199f385"} Sep 17 19:56:13.085721 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:13.085648 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:13.102411 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:13.102365 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" podStartSLOduration=2.102349856 podStartE2EDuration="2.102349856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:56:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:56:13.10159296 +0000 UTC m=+2981.363466391" watchObservedRunningTime="2026-09-17 19:56:13.102349856 +0000 UTC m=+2981.364223263" Sep 17 19:56:13.190451 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:13.190426 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q89g2_76086d58-ef7d-4a2c-9614-c15931bac3df/node-ca/0.log" Sep 17 19:56:13.984658 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:13.984629 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-544cd45fb-8wb98_7213bf71-54de-4fc3-8fa0-89b2e1d45f51/router/0.log" Sep 17 19:56:14.363330 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:14.363260 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qlb49_826c88c0-6862-4bc7-ad50-51c7365ded8b/serve-healthcheck-canary/0.log" Sep 17 19:56:14.803022 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:14.802994 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-mml6g_0280dd0f-999f-44b2-b052-4652b01c394c/insights-operator/0.log" Sep 17 19:56:14.803626 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:14.803613 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-mml6g_0280dd0f-999f-44b2-b052-4652b01c394c/insights-operator/1.log" Sep 17 19:56:14.826206 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:14.826183 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6lp9m_cfeffc39-8808-4e44-b91c-7986b85a38a6/kube-rbac-proxy/0.log" Sep 17 19:56:14.850016 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:14.849987 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6lp9m_cfeffc39-8808-4e44-b91c-7986b85a38a6/exporter/0.log" Sep 17 19:56:14.874450 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:14.874431 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6lp9m_cfeffc39-8808-4e44-b91c-7986b85a38a6/extractor/0.log" Sep 17 19:56:17.561404 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:17.561370 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-clngb_9794b921-7759-4b84-800d-1069b28e4601/s3-init/0.log" Sep 17 19:56:17.578700 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:17.578680 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-clngb_9794b921-7759-4b84-800d-1069b28e4601/download-hf-model/0.log" Sep 17 19:56:19.099528 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:19.099501 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-7v2cb" Sep 17 19:56:23.682156 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:23.682129 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9wcr_8b46af12-297a-4da0-b3f3-730424604854/kube-multus-additional-cni-plugins/0.log" Sep 17 19:56:23.705055 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:23.705033 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9wcr_8b46af12-297a-4da0-b3f3-730424604854/egress-router-binary-copy/0.log" Sep 17 19:56:23.729222 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:23.729204 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9wcr_8b46af12-297a-4da0-b3f3-730424604854/cni-plugins/0.log" Sep 17 19:56:23.754924 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:23.754904 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9wcr_8b46af12-297a-4da0-b3f3-730424604854/bond-cni-plugin/0.log" Sep 17 19:56:23.780979 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:23.780962 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9wcr_8b46af12-297a-4da0-b3f3-730424604854/routeoverride-cni/0.log" Sep 17 19:56:23.804294 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:23.804277 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9wcr_8b46af12-297a-4da0-b3f3-730424604854/whereabouts-cni-bincopy/0.log" Sep 17 19:56:23.827324 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:23.827309 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f9wcr_8b46af12-297a-4da0-b3f3-730424604854/whereabouts-cni/0.log" Sep 17 19:56:24.054347 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:24.054318 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-r5nkm_5ce6506b-9752-4fbe-9622-29ecf358b465/kube-multus/0.log" Sep 17 19:56:24.126809 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:24.126789 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6mspz_431387c9-5a3c-44d5-bad2-94a08bc73700/network-metrics-daemon/0.log" Sep 17 19:56:24.147777 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:24.147758 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6mspz_431387c9-5a3c-44d5-bad2-94a08bc73700/kube-rbac-proxy/0.log" Sep 17 19:56:25.375698 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:25.375668 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5854_c13b5773-a330-4e28-8c36-dbed2847dd86/ovn-controller/0.log" Sep 17 19:56:25.399341 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:25.399308 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5854_c13b5773-a330-4e28-8c36-dbed2847dd86/ovn-acl-logging/0.log" Sep 17 19:56:25.425870 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:25.425850 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5854_c13b5773-a330-4e28-8c36-dbed2847dd86/kube-rbac-proxy-node/0.log" Sep 17 19:56:25.455131 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:25.455086 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5854_c13b5773-a330-4e28-8c36-dbed2847dd86/kube-rbac-proxy-ovn-metrics/0.log" Sep 17 19:56:25.475704 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:25.475688 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5854_c13b5773-a330-4e28-8c36-dbed2847dd86/northd/0.log" Sep 17 19:56:25.498000 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:25.497980 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5854_c13b5773-a330-4e28-8c36-dbed2847dd86/nbdb/0.log" Sep 17 19:56:25.521496 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:25.521475 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5854_c13b5773-a330-4e28-8c36-dbed2847dd86/sbdb/0.log" Sep 17 19:56:25.557663 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:25.557646 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m5854_c13b5773-a330-4e28-8c36-dbed2847dd86/ovnkube-controller/0.log" Sep 17 19:56:26.640145 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:26.640119 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-z7q6d_357e149f-654a-4188-9229-1a440c1c30a9/network-check-target-container/0.log" Sep 17 19:56:27.627220 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:27.627192 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-69wbb_5827bc3d-1d3b-4b60-9217-ffe0998fc733/iptables-alerter/0.log" Sep 17 19:56:28.419675 ip-10-0-135-16 kubenswrapper[2563]: I0917 19:56:28.419650 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-dc7vl_532a9820-a0c8-4dec-a697-9e7d15057df6/tuned/0.log"