Apr 16 18:15:35.477941 ip-10-0-138-22 systemd[1]: Starting Kubernetes Kubelet... Apr 16 18:15:35.950219 ip-10-0-138-22 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 18:15:35.950219 ip-10-0-138-22 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Apr 16 18:15:35.950219 ip-10-0-138-22 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 18:15:35.950219 ip-10-0-138-22 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Apr 16 18:15:35.950219 ip-10-0-138-22 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 18:15:35.953026 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.952938 2577 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Apr 16 18:15:35.958441 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958413 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 18:15:35.958441 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958436 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 18:15:35.958441 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958441 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 18:15:35.958441 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958445 2577 feature_gate.go:328] unrecognized feature gate: Example Apr 16 18:15:35.958441 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958449 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958455 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958458 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958461 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958464 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958467 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958470 2577 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958473 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958476 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958479 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958482 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958484 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958487 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958490 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958494 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958498 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958501 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958503 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958508 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958510 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 18:15:35.958686 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958513 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958515 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958518 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958521 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958523 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958526 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958528 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958531 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958534 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958536 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958538 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958541 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958543 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958546 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958548 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958551 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958554 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958556 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958559 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958562 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 18:15:35.959150 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958564 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958566 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958569 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958572 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958575 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958578 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958580 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958582 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958585 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958587 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958590 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958592 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958595 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958597 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958599 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958602 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958604 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958607 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958609 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958612 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 18:15:35.959700 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958615 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958617 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958620 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958641 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958647 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958651 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958656 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958661 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958665 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958668 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958670 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958679 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958681 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958684 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958688 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958691 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958693 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958696 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958699 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958702 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 18:15:35.960184 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958704 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.958707 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959148 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959154 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959157 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959160 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959164 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959166 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959169 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959172 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959175 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959177 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959180 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959183 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959186 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959188 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959191 2577 feature_gate.go:328] unrecognized feature gate: Example Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959193 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959196 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959198 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 18:15:35.960671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959201 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959204 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959206 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959209 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959211 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959214 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959217 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959220 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959223 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959227 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959230 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959233 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959235 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959238 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959240 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959243 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959245 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959248 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959250 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 18:15:35.961182 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959253 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959255 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959258 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959260 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959262 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959266 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959269 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959271 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959274 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959276 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959278 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959281 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959284 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959286 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959289 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959292 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959294 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959297 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959299 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959302 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 18:15:35.961701 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959306 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959308 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959311 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959313 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959316 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959318 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959321 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959323 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959326 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959329 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959331 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959333 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959336 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959339 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959342 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959344 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959346 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959350 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959353 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959355 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 18:15:35.962193 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959358 2577 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959360 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959364 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959368 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959372 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959374 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959377 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959380 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.959382 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960689 2577 flags.go:64] FLAG: --address="0.0.0.0" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960699 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960706 2577 flags.go:64] FLAG: --anonymous-auth="true" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960711 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960721 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960725 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960735 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960739 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960742 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960745 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960749 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960752 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960755 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Apr 16 18:15:35.962709 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960758 2577 flags.go:64] FLAG: --cgroup-root="" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960761 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960763 2577 flags.go:64] FLAG: --client-ca-file="" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960766 2577 flags.go:64] FLAG: --cloud-config="" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960769 2577 flags.go:64] FLAG: --cloud-provider="external" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960772 2577 flags.go:64] FLAG: --cluster-dns="[]" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960777 2577 flags.go:64] FLAG: --cluster-domain="" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960780 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960783 2577 flags.go:64] FLAG: --config-dir="" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960786 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960789 2577 flags.go:64] FLAG: --container-log-max-files="5" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960793 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960796 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960800 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960804 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960807 2577 flags.go:64] FLAG: --contention-profiling="false" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960810 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960813 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960816 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960818 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960823 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960826 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960829 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960832 2577 flags.go:64] FLAG: --enable-load-reader="false" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960835 2577 flags.go:64] FLAG: --enable-server="true" Apr 16 18:15:35.963236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960838 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960843 2577 flags.go:64] FLAG: --event-burst="100" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960846 2577 flags.go:64] FLAG: --event-qps="50" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960849 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960851 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960855 2577 flags.go:64] FLAG: --eviction-hard="" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960858 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960861 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960864 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960867 2577 flags.go:64] FLAG: --eviction-soft="" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960870 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960873 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960876 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960879 2577 flags.go:64] FLAG: --experimental-mounter-path="" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960882 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960886 2577 flags.go:64] FLAG: --fail-swap-on="true" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960889 2577 flags.go:64] FLAG: --feature-gates="" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960897 2577 flags.go:64] FLAG: --file-check-frequency="20s" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960900 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960903 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960907 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960910 2577 flags.go:64] FLAG: --healthz-port="10248" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960913 2577 flags.go:64] FLAG: --help="false" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960916 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-138-22.ec2.internal" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960919 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Apr 16 18:15:35.963861 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960922 2577 flags.go:64] FLAG: --http-check-frequency="20s" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960925 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960928 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960932 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960934 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960937 2577 flags.go:64] FLAG: --image-service-endpoint="" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960940 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960943 2577 flags.go:64] FLAG: --kube-api-burst="100" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960946 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960949 2577 flags.go:64] FLAG: --kube-api-qps="50" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960952 2577 flags.go:64] FLAG: --kube-reserved="" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960955 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960958 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960961 2577 flags.go:64] FLAG: --kubelet-cgroups="" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960964 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960967 2577 flags.go:64] FLAG: --lock-file="" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960969 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960972 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960975 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960980 2577 flags.go:64] FLAG: --log-json-split-stream="false" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960983 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960987 2577 flags.go:64] FLAG: --log-text-split-stream="false" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960990 2577 flags.go:64] FLAG: --logging-format="text" Apr 16 18:15:35.964452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960993 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960996 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.960999 2577 flags.go:64] FLAG: --manifest-url="" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961002 2577 flags.go:64] FLAG: --manifest-url-header="" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961007 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961010 2577 flags.go:64] FLAG: --max-open-files="1000000" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961014 2577 flags.go:64] FLAG: --max-pods="110" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961017 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961020 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961022 2577 flags.go:64] FLAG: --memory-manager-policy="None" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961025 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961028 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961031 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961034 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961042 2577 flags.go:64] FLAG: --node-status-max-images="50" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961045 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961048 2577 flags.go:64] FLAG: --oom-score-adj="-999" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961052 2577 flags.go:64] FLAG: --pod-cidr="" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961054 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dc76bab72f320de3d4105c90d73c4fb139c09e20ce0fa8dcbc0cb59920d27dec" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961060 2577 flags.go:64] FLAG: --pod-manifest-path="" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961063 2577 flags.go:64] FLAG: --pod-max-pids="-1" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961066 2577 flags.go:64] FLAG: --pods-per-core="0" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961069 2577 flags.go:64] FLAG: --port="10250" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961072 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Apr 16 18:15:35.965017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961075 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d3499c5442bc4636" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961079 2577 flags.go:64] FLAG: --qos-reserved="" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961081 2577 flags.go:64] FLAG: --read-only-port="10255" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961084 2577 flags.go:64] FLAG: --register-node="true" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961087 2577 flags.go:64] FLAG: --register-schedulable="true" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961090 2577 flags.go:64] FLAG: --register-with-taints="" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961094 2577 flags.go:64] FLAG: --registry-burst="10" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961097 2577 flags.go:64] FLAG: --registry-qps="5" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961099 2577 flags.go:64] FLAG: --reserved-cpus="" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961102 2577 flags.go:64] FLAG: --reserved-memory="" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961106 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961109 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961112 2577 flags.go:64] FLAG: --rotate-certificates="false" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961115 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961117 2577 flags.go:64] FLAG: --runonce="false" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961120 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961123 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961126 2577 flags.go:64] FLAG: --seccomp-default="false" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961129 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961131 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961134 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961137 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961140 2577 flags.go:64] FLAG: --storage-driver-password="root" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961143 2577 flags.go:64] FLAG: --storage-driver-secure="false" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961146 2577 flags.go:64] FLAG: --storage-driver-table="stats" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961148 2577 flags.go:64] FLAG: --storage-driver-user="root" Apr 16 18:15:35.965601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961151 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961154 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961157 2577 flags.go:64] FLAG: --system-cgroups="" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961163 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961168 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961171 2577 flags.go:64] FLAG: --tls-cert-file="" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961174 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961178 2577 flags.go:64] FLAG: --tls-min-version="" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961181 2577 flags.go:64] FLAG: --tls-private-key-file="" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961183 2577 flags.go:64] FLAG: --topology-manager-policy="none" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961186 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961189 2577 flags.go:64] FLAG: --topology-manager-scope="container" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961193 2577 flags.go:64] FLAG: --v="2" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961197 2577 flags.go:64] FLAG: --version="false" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961201 2577 flags.go:64] FLAG: --vmodule="" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961205 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.961209 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961326 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961330 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961333 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961336 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961339 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961341 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 18:15:35.966243 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961344 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961347 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961349 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961352 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961355 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961357 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961360 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961362 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961365 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961367 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961370 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961374 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961377 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961380 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961382 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961385 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961388 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961390 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961393 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961395 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 18:15:35.966815 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961398 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961401 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961403 2577 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961406 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961409 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961412 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961414 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961417 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961419 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961423 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961427 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961429 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961432 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961435 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961437 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961440 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961442 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961444 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961447 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961449 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 18:15:35.967359 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961466 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961470 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961473 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961476 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961480 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961483 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961486 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961488 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961491 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961493 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961496 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961498 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961501 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961503 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961506 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961508 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961510 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961513 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961516 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961518 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 18:15:35.967901 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961521 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961523 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961525 2577 feature_gate.go:328] unrecognized feature gate: Example Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961528 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961531 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961533 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961536 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961539 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961541 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961543 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961546 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961548 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961551 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961553 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961556 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961558 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961562 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961565 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961569 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 18:15:35.968396 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.961575 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 18:15:35.968873 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.962206 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 18:15:35.968873 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.968848 2577 server.go:530] "Kubelet version" kubeletVersion="v1.33.9" Apr 16 18:15:35.968873 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.968868 2577 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Apr 16 18:15:35.968964 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968941 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 18:15:35.968964 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968946 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 18:15:35.968964 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968950 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 18:15:35.968964 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968953 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 18:15:35.968964 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968956 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 18:15:35.968964 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968960 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 18:15:35.968964 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968964 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968970 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968973 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968976 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968979 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968981 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968985 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968987 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968990 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968993 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968996 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.968998 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969001 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969003 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969006 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969008 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969010 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969014 2577 feature_gate.go:328] unrecognized feature gate: Example Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969016 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969020 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 18:15:35.969134 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969024 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969027 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969029 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969032 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969035 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969038 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969041 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969043 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969045 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969048 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969051 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969053 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969056 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969058 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969062 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969064 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969067 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969069 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969072 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 18:15:35.969640 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969075 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969077 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969079 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969082 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969085 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969087 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969090 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969092 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969095 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969097 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969099 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969103 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969105 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969107 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969110 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969112 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969114 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969117 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969119 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 18:15:35.970106 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969123 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969125 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969128 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969130 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969132 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969135 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969138 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969141 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969144 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969147 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969149 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969152 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969154 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969157 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969159 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969162 2577 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969164 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969166 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969169 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969172 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 18:15:35.970581 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969174 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969177 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.969182 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969282 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969286 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969289 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969292 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969295 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969298 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969301 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969304 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969307 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969309 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969313 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969316 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 18:15:35.971119 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969319 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969322 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969325 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969327 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969330 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969333 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969335 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969337 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969340 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969342 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969345 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969347 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969350 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969352 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969355 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969357 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969360 2577 feature_gate.go:328] unrecognized feature gate: Example Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969362 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969365 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 18:15:35.971494 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969369 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969372 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969375 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969378 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969380 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969383 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969386 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969388 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969391 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969393 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969396 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969398 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969407 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969410 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969413 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969415 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969418 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969420 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969423 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969425 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 18:15:35.972040 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969428 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969430 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969433 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969436 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969439 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969441 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969443 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969446 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969448 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969451 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969453 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969455 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969458 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969460 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969463 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969465 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969468 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969470 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969473 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 18:15:35.972523 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969476 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969478 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969480 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969483 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969485 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969487 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969491 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969493 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969496 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969498 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969501 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969503 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969506 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969509 2577 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969511 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:35.969514 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 18:15:35.972993 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.969519 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 18:15:35.973548 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.970341 2577 server.go:962] "Client rotation is on, will bootstrap in background" Apr 16 18:15:35.973548 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.973110 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Apr 16 18:15:35.974243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.974231 2577 server.go:1019] "Starting client certificate rotation" Apr 16 18:15:35.974367 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.974346 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 16 18:15:35.974429 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:35.974403 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 16 18:15:36.003015 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.002979 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 16 18:15:36.006404 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.006368 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 16 18:15:36.019534 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.019513 2577 log.go:25] "Validated CRI v1 runtime API" Apr 16 18:15:36.025136 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.025110 2577 log.go:25] "Validated CRI v1 image API" Apr 16 18:15:36.029561 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.029542 2577 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Apr 16 18:15:36.030729 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.030703 2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 16 18:15:36.032403 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.032380 2577 fs.go:135] Filesystem UUIDs: map[442f49a5-7c93-4445-be6d-f2a36b65a04a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 dc7ecc8f-421b-43d6-8c9c-b2e52591077e:/dev/nvme0n1p4] Apr 16 18:15:36.032493 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.032402 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Apr 16 18:15:36.038271 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.038150 2577 manager.go:217] Machine: {Timestamp:2026-04-16 18:15:36.03623614 +0000 UTC m=+0.429617388 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100059 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22d0f44a955a28f7ee8dd8b7aec413 SystemUUID:ec22d0f4-4a95-5a28-f7ee-8dd8b7aec413 BootID:e6b4d8b5-6822-4489-a57d-8c41b492c16b Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6098944 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:81:7d:ea:a6:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:81:7d:ea:a6:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:05:68:dc:6b:f3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Apr 16 18:15:36.038271 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.038258 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Apr 16 18:15:36.038439 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.038365 2577 manager.go:233] Version: {KernelVersion:5.14.0-570.104.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260401-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Apr 16 18:15:36.039259 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.039230 2577 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Apr 16 18:15:36.039417 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.039260 2577 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-22.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Apr 16 18:15:36.039503 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.039432 2577 topology_manager.go:138] "Creating topology manager with none policy" Apr 16 18:15:36.039503 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.039444 2577 container_manager_linux.go:306] "Creating device plugin manager" Apr 16 18:15:36.039503 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.039468 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 16 18:15:36.039503 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.039488 2577 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 16 18:15:36.041255 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.041241 2577 state_mem.go:36] "Initialized new in-memory state store" Apr 16 18:15:36.041389 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.041378 2577 server.go:1267] "Using root directory" path="/var/lib/kubelet" Apr 16 18:15:36.043601 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.043589 2577 kubelet.go:491] "Attempting to sync node with API server" Apr 16 18:15:36.043675 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.043616 2577 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Apr 16 18:15:36.043675 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.043654 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Apr 16 18:15:36.043675 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.043669 2577 kubelet.go:397] "Adding apiserver pod source" Apr 16 18:15:36.043801 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.043682 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Apr 16 18:15:36.044819 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.044805 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 16 18:15:36.044887 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.044832 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 16 18:15:36.048033 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.048015 2577 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.10-2.rhaos4.20.gita4d0894.el9" apiVersion="v1" Apr 16 18:15:36.049838 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.049816 2577 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Apr 16 18:15:36.051136 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051125 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Apr 16 18:15:36.051181 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051142 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Apr 16 18:15:36.051181 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051148 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Apr 16 18:15:36.051181 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051154 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Apr 16 18:15:36.051181 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051160 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Apr 16 18:15:36.051181 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051166 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Apr 16 18:15:36.051181 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051172 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Apr 16 18:15:36.051181 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051177 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Apr 16 18:15:36.051181 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051184 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Apr 16 18:15:36.051389 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051190 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Apr 16 18:15:36.051389 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051199 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Apr 16 18:15:36.051389 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.051208 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Apr 16 18:15:36.053093 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.053079 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Apr 16 18:15:36.053093 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.053094 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Apr 16 18:15:36.054858 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.054835 2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Apr 16 18:15:36.054935 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.054877 2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-22.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Apr 16 18:15:36.054935 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.054924 2577 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-22.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Apr 16 18:15:36.056651 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.056620 2577 watchdog_linux.go:99] "Systemd watchdog is not enabled" Apr 16 18:15:36.056696 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.056675 2577 server.go:1295] "Started kubelet" Apr 16 18:15:36.056793 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.056757 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Apr 16 18:15:36.056838 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.056765 2577 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Apr 16 18:15:36.056895 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.056848 2577 server_v1.go:47] "podresources" method="list" useActivePods=true Apr 16 18:15:36.057486 ip-10-0-138-22 systemd[1]: Started Kubernetes Kubelet. Apr 16 18:15:36.062730 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.062705 2577 server.go:317] "Adding debug handlers to kubelet server" Apr 16 18:15:36.062814 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.062751 2577 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Apr 16 18:15:36.063330 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.063305 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s84ll" Apr 16 18:15:36.066815 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.066757 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Apr 16 18:15:36.066815 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.066782 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Apr 16 18:15:36.067383 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.067361 2577 volume_manager.go:295] "The desired_state_of_world populator starts" Apr 16 18:15:36.067383 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.067381 2577 volume_manager.go:297] "Starting Kubelet Volume Manager" Apr 16 18:15:36.067507 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.067440 2577 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Apr 16 18:15:36.067768 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.067611 2577 reconstruct.go:97] "Volume reconstruction finished" Apr 16 18:15:36.067768 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.067756 2577 reconciler.go:26] "Reconciler: start to sync state" Apr 16 18:15:36.069420 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.069189 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s84ll" Apr 16 18:15:36.069738 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.069715 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:36.070253 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.070217 2577 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Apr 16 18:15:36.070341 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.070328 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Apr 16 18:15:36.070383 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.070344 2577 factory.go:55] Registering systemd factory Apr 16 18:15:36.070383 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.070350 2577 factory.go:223] Registration of the systemd container factory successfully Apr 16 18:15:36.070587 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.070576 2577 factory.go:153] Registering CRI-O factory Apr 16 18:15:36.070587 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.070589 2577 factory.go:223] Registration of the crio container factory successfully Apr 16 18:15:36.070699 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.070613 2577 factory.go:103] Registering Raw factory Apr 16 18:15:36.070699 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.070646 2577 manager.go:1196] Started watching for new ooms in manager Apr 16 18:15:36.071240 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.071229 2577 manager.go:319] Starting recovery of all containers Apr 16 18:15:36.075123 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.075101 2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 18:15:36.078129 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.078105 2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-22.ec2.internal\" not found" node="ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.081825 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.081675 2577 manager.go:324] Recovery completed Apr 16 18:15:36.085929 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.085912 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 18:15:36.088349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.088333 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientMemory" Apr 16 18:15:36.088438 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.088366 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 18:15:36.088438 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.088386 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientPID" Apr 16 18:15:36.088924 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.088913 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Apr 16 18:15:36.088985 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.088924 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Apr 16 18:15:36.088985 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.088940 2577 state_mem.go:36] "Initialized new in-memory state store" Apr 16 18:15:36.091034 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.091022 2577 policy_none.go:49] "None policy: Start" Apr 16 18:15:36.091075 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.091047 2577 memory_manager.go:186] "Starting memorymanager" policy="None" Apr 16 18:15:36.091469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.091461 2577 state_mem.go:35] "Initializing new in-memory state store" Apr 16 18:15:36.138291 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.138272 2577 manager.go:341] "Starting Device Plugin manager" Apr 16 18:15:36.139610 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.138314 2577 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Apr 16 18:15:36.139610 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.138328 2577 server.go:85] "Starting device plugin registration server" Apr 16 18:15:36.139610 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.138606 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Apr 16 18:15:36.139610 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.138618 2577 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Apr 16 18:15:36.139610 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.138741 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Apr 16 18:15:36.139610 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.138822 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Apr 16 18:15:36.139610 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.138831 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Apr 16 18:15:36.139610 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.139362 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Apr 16 18:15:36.139610 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.139396 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:36.207812 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.207727 2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Apr 16 18:15:36.209088 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.209066 2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Apr 16 18:15:36.209153 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.209106 2577 status_manager.go:230] "Starting to sync pod status with apiserver" Apr 16 18:15:36.209153 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.209133 2577 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Apr 16 18:15:36.209153 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.209144 2577 kubelet.go:2451] "Starting kubelet main sync loop" Apr 16 18:15:36.209242 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.209186 2577 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Apr 16 18:15:36.213215 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.213191 2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 18:15:36.239119 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.239093 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 18:15:36.240057 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.240041 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientMemory" Apr 16 18:15:36.240130 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.240070 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 18:15:36.240130 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.240083 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientPID" Apr 16 18:15:36.240130 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.240113 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.249056 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.249036 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.249105 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.249064 2577 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-22.ec2.internal\": node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:36.265084 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.265067 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:36.309549 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.309506 2577 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-22.ec2.internal"] Apr 16 18:15:36.309611 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.309598 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 18:15:36.311201 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.311187 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientMemory" Apr 16 18:15:36.311274 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.311218 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 18:15:36.311274 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.311236 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientPID" Apr 16 18:15:36.312439 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.312426 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 18:15:36.312599 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.312584 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.312646 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.312616 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 18:15:36.313203 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.313185 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientMemory" Apr 16 18:15:36.313309 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.313214 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientMemory" Apr 16 18:15:36.313309 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.313239 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 18:15:36.313309 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.313250 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientPID" Apr 16 18:15:36.313309 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.313217 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 18:15:36.313309 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.313304 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientPID" Apr 16 18:15:36.314794 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.314780 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.314862 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.314807 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 18:15:36.315505 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.315481 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientMemory" Apr 16 18:15:36.315584 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.315510 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 18:15:36.315584 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.315521 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeHasSufficientPID" Apr 16 18:15:36.331271 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.331246 2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-22.ec2.internal\" not found" node="ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.334943 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.334925 2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-22.ec2.internal\" not found" node="ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.366069 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.366041 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:36.370420 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.370397 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3c5a36ec21f8e94d847668d8289fd26-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal\" (UID: \"d3c5a36ec21f8e94d847668d8289fd26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.370490 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.370429 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3c5a36ec21f8e94d847668d8289fd26-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal\" (UID: \"d3c5a36ec21f8e94d847668d8289fd26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.370490 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.370446 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9651c74f7f561de9ee59b0475859f99e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-22.ec2.internal\" (UID: \"9651c74f7f561de9ee59b0475859f99e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.466522 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.466431 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:36.470764 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.470745 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3c5a36ec21f8e94d847668d8289fd26-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal\" (UID: \"d3c5a36ec21f8e94d847668d8289fd26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.470838 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.470775 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3c5a36ec21f8e94d847668d8289fd26-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal\" (UID: \"d3c5a36ec21f8e94d847668d8289fd26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.470838 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.470794 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9651c74f7f561de9ee59b0475859f99e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-22.ec2.internal\" (UID: \"9651c74f7f561de9ee59b0475859f99e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.470925 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.470853 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3c5a36ec21f8e94d847668d8289fd26-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal\" (UID: \"d3c5a36ec21f8e94d847668d8289fd26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.470925 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.470883 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9651c74f7f561de9ee59b0475859f99e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-22.ec2.internal\" (UID: \"9651c74f7f561de9ee59b0475859f99e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.470925 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.470858 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3c5a36ec21f8e94d847668d8289fd26-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal\" (UID: \"d3c5a36ec21f8e94d847668d8289fd26\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.566862 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.566828 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:36.634502 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.634475 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.637372 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.637353 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-22.ec2.internal" Apr 16 18:15:36.667935 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.667902 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:36.768432 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.768347 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:36.868772 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.868745 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:36.969384 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:36.969353 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:36.973545 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.973526 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Apr 16 18:15:36.973714 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.973693 2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 16 18:15:36.973759 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.973693 2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 16 18:15:36.992852 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:36.992821 2577 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 18:15:37.067144 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.067117 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Apr 16 18:15:37.070351 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:37.070332 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:37.071441 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.071404 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-04-15 18:10:36 +0000 UTC" deadline="2027-12-29 19:02:43.926092052 +0000 UTC" Apr 16 18:15:37.071489 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.071441 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14928h47m6.854653647s" Apr 16 18:15:37.075746 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.075727 2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 16 18:15:37.107599 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.107567 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-42vvs" Apr 16 18:15:37.113779 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.113756 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-42vvs" Apr 16 18:15:37.156904 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:37.156857 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c5a36ec21f8e94d847668d8289fd26.slice/crio-5513232705bd9a3cdb0f84a78c8823c4863534dbc986fd42547a8dd58d034a17 WatchSource:0}: Error finding container 5513232705bd9a3cdb0f84a78c8823c4863534dbc986fd42547a8dd58d034a17: Status 404 returned error can't find the container with id 5513232705bd9a3cdb0f84a78c8823c4863534dbc986fd42547a8dd58d034a17 Apr 16 18:15:37.157152 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:37.157138 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9651c74f7f561de9ee59b0475859f99e.slice/crio-7951ade42b54f1d6f47347653462baecf02280107936232e8f28bd6dd3af9e8a WatchSource:0}: Error finding container 7951ade42b54f1d6f47347653462baecf02280107936232e8f28bd6dd3af9e8a: Status 404 returned error can't find the container with id 7951ade42b54f1d6f47347653462baecf02280107936232e8f28bd6dd3af9e8a Apr 16 18:15:37.161663 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.161645 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 18:15:37.171290 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:37.171266 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:37.212416 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.212351 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" event={"ID":"d3c5a36ec21f8e94d847668d8289fd26","Type":"ContainerStarted","Data":"5513232705bd9a3cdb0f84a78c8823c4863534dbc986fd42547a8dd58d034a17"} Apr 16 18:15:37.213255 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.213235 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-22.ec2.internal" event={"ID":"9651c74f7f561de9ee59b0475859f99e","Type":"ContainerStarted","Data":"7951ade42b54f1d6f47347653462baecf02280107936232e8f28bd6dd3af9e8a"} Apr 16 18:15:37.271376 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:37.271350 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-22.ec2.internal\" not found" Apr 16 18:15:37.271614 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.271602 2577 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 18:15:37.367329 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.367242 2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" Apr 16 18:15:37.399050 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.399019 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 16 18:15:37.400104 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.400086 2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-22.ec2.internal" Apr 16 18:15:37.420028 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.420004 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 16 18:15:37.860494 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:37.860419 2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 18:15:38.044244 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.044211 2577 apiserver.go:52] "Watching apiserver" Apr 16 18:15:38.053572 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.053545 2577 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Apr 16 18:15:38.053965 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.053942 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-69k7d","openshift-multus/network-metrics-daemon-lp8x6","openshift-network-diagnostics/network-check-target-zf6t5","openshift-network-operator/iptables-alerter-25ptb","openshift-ovn-kubernetes/ovnkube-node-wgfh9","kube-system/konnectivity-agent-vhpvh","kube-system/kube-apiserver-proxy-ip-10-0-138-22.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q","openshift-dns/node-resolver-kdfqc","openshift-image-registry/node-ca-wh4nz","openshift-multus/multus-additional-cni-plugins-59lzl","openshift-cluster-node-tuning-operator/tuned-9csqh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal"] Apr 16 18:15:38.056429 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.056410 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wh4nz" Apr 16 18:15:38.058394 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.058373 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Apr 16 18:15:38.058394 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.058382 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Apr 16 18:15:38.058536 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.058375 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Apr 16 18:15:38.058821 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.058802 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-q9cpn\"" Apr 16 18:15:38.059067 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.059050 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:38.059160 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.059133 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:15:38.061253 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.061232 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:38.061361 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.061326 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:38.063768 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.063298 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-25ptb" Apr 16 18:15:38.064951 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.064928 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Apr 16 18:15:38.065042 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.065010 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Apr 16 18:15:38.065042 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.065022 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Apr 16 18:15:38.065214 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.065194 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r9x9j\"" Apr 16 18:15:38.065603 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.065585 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.067544 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.067525 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Apr 16 18:15:38.067885 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.067749 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Apr 16 18:15:38.067885 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.067772 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8rdb2\"" Apr 16 18:15:38.067885 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.067786 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Apr 16 18:15:38.068074 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.067961 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Apr 16 18:15:38.068129 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.068076 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Apr 16 18:15:38.068181 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.068139 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Apr 16 18:15:38.070269 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.070244 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:38.070359 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.070339 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.072098 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.072078 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Apr 16 18:15:38.072190 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.072122 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Apr 16 18:15:38.072405 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.072387 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-slcpk\"" Apr 16 18:15:38.072507 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.072418 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Apr 16 18:15:38.072755 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.072734 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Apr 16 18:15:38.072824 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.072766 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-r6pjn\"" Apr 16 18:15:38.072879 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.072870 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Apr 16 18:15:38.073446 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.073429 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kdfqc" Apr 16 18:15:38.074981 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.074966 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8n2dn\"" Apr 16 18:15:38.075229 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.075199 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Apr 16 18:15:38.075299 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.075280 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Apr 16 18:15:38.075739 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.075693 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.077492 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.077473 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Apr 16 18:15:38.078032 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.078012 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Apr 16 18:15:38.078126 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.078038 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vcbcz\"" Apr 16 18:15:38.078249 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.078228 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Apr 16 18:15:38.078444 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.078424 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Apr 16 18:15:38.078505 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.078469 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.078860 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-cni-netd\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.078924 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.079048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/109afb78-2acc-4972-9604-32fb10a7ae6b-serviceca\") pod \"node-ca-wh4nz\" (UID: \"109afb78-2acc-4972-9604-32fb10a7ae6b\") " pod="openshift-image-registry/node-ca-wh4nz" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.079128 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-var-lib-openvswitch\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.079154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-run-ovn\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.079179 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-env-overrides\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.079267 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-kubelet\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.079556 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-systemd-units\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.079760 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-cni-bin\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.079868 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87f2496f-9291-4513-812e-23f4132ec75c-agent-certs\") pod \"konnectivity-agent-vhpvh\" (UID: \"87f2496f-9291-4513-812e-23f4132ec75c\") " pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.079929 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87f2496f-9291-4513-812e-23f4132ec75c-konnectivity-ca\") pod \"konnectivity-agent-vhpvh\" (UID: \"87f2496f-9291-4513-812e-23f4132ec75c\") " pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:38.080078 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080044 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/109afb78-2acc-4972-9604-32fb10a7ae6b-host\") pod \"node-ca-wh4nz\" (UID: \"109afb78-2acc-4972-9604-32fb10a7ae6b\") " pod="openshift-image-registry/node-ca-wh4nz" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080089 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7pq5\" (UniqueName: \"kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5\") pod \"network-check-target-zf6t5\" (UID: \"9792d3fc-f6a7-4d56-a906-b650dea53fb2\") " pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080153 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-slash\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080204 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-run-systemd\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080240 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-run-openvswitch\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080284 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-ovnkube-script-lib\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080340 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnc2g\" (UniqueName: \"kubernetes.io/projected/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-kube-api-access-qnc2g\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080381 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/261d6b5b-fcc0-49a2-a408-af233d788aa3-iptables-alerter-script\") pod \"iptables-alerter-25ptb\" (UID: \"261d6b5b-fcc0-49a2-a408-af233d788aa3\") " pod="openshift-network-operator/iptables-alerter-25ptb" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080409 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jrrz\" (UniqueName: \"kubernetes.io/projected/261d6b5b-fcc0-49a2-a408-af233d788aa3-kube-api-access-6jrrz\") pod \"iptables-alerter-25ptb\" (UID: \"261d6b5b-fcc0-49a2-a408-af233d788aa3\") " pod="openshift-network-operator/iptables-alerter-25ptb" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080469 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-run-netns\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-log-socket\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080583 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-ovn-node-metrics-cert\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.080731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080658 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfc5p\" (UniqueName: \"kubernetes.io/projected/5609e075-d6f5-4642-a711-20f608288876-kube-api-access-vfc5p\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:38.081233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080749 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-ovnkube-config\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.081233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080795 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:38.081233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080865 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/261d6b5b-fcc0-49a2-a408-af233d788aa3-host-slash\") pod \"iptables-alerter-25ptb\" (UID: \"261d6b5b-fcc0-49a2-a408-af233d788aa3\") " pod="openshift-network-operator/iptables-alerter-25ptb" Apr 16 18:15:38.081233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080936 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-etc-openvswitch\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.081233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.080980 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-run-ovn-kubernetes\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.081233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.081030 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn5jr\" (UniqueName: \"kubernetes.io/projected/109afb78-2acc-4972-9604-32fb10a7ae6b-kube-api-access-xn5jr\") pod \"node-ca-wh4nz\" (UID: \"109afb78-2acc-4972-9604-32fb10a7ae6b\") " pod="openshift-image-registry/node-ca-wh4nz" Apr 16 18:15:38.081233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.081073 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-node-log\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.081233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.081110 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Apr 16 18:15:38.081233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.081139 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Apr 16 18:15:38.081676 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.081507 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-72ndk\"" Apr 16 18:15:38.082517 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.082395 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.084881 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.084577 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dpdl8\"" Apr 16 18:15:38.084881 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.084585 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Apr 16 18:15:38.084881 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.084589 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Apr 16 18:15:38.114696 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.114603 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-15 18:10:37 +0000 UTC" deadline="2027-09-10 04:58:37.545679411 +0000 UTC" Apr 16 18:15:38.114696 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.114662 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12274h42m59.431021705s" Apr 16 18:15:38.168539 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.168504 2577 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Apr 16 18:15:38.181441 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181412 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/109afb78-2acc-4972-9604-32fb10a7ae6b-host\") pod \"node-ca-wh4nz\" (UID: \"109afb78-2acc-4972-9604-32fb10a7ae6b\") " pod="openshift-image-registry/node-ca-wh4nz" Apr 16 18:15:38.181644 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181453 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-slash\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.181644 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181485 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-run-systemd\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.181644 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181543 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-run-systemd\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.181644 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181543 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-slash\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.181644 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181566 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-run-openvswitch\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.181644 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181593 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-run-openvswitch\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.181644 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181601 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qnc2g\" (UniqueName: \"kubernetes.io/projected/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-kube-api-access-qnc2g\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.181978 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181649 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6jrrz\" (UniqueName: \"kubernetes.io/projected/261d6b5b-fcc0-49a2-a408-af233d788aa3-kube-api-access-6jrrz\") pod \"iptables-alerter-25ptb\" (UID: \"261d6b5b-fcc0-49a2-a408-af233d788aa3\") " pod="openshift-network-operator/iptables-alerter-25ptb" Apr 16 18:15:38.181978 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181675 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-run-netns\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.181978 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181700 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-ovn-node-metrics-cert\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.181978 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181728 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8l4s\" (UniqueName: \"kubernetes.io/projected/59928e2c-472c-4289-b77d-213f2811bb82-kube-api-access-d8l4s\") pod \"node-resolver-kdfqc\" (UID: \"59928e2c-472c-4289-b77d-213f2811bb82\") " pod="openshift-dns/node-resolver-kdfqc" Apr 16 18:15:38.181978 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181766 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-run-netns\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.181978 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181816 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/109afb78-2acc-4972-9604-32fb10a7ae6b-host\") pod \"node-ca-wh4nz\" (UID: \"109afb78-2acc-4972-9604-32fb10a7ae6b\") " pod="openshift-image-registry/node-ca-wh4nz" Apr 16 18:15:38.181978 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181861 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-sysctl-d\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.181978 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181898 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfc5p\" (UniqueName: \"kubernetes.io/projected/5609e075-d6f5-4642-a711-20f608288876-kube-api-access-vfc5p\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:38.181978 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181942 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-socket-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.181962 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-etc-openvswitch\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182010 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-device-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182034 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59928e2c-472c-4289-b77d-213f2811bb82-tmp-dir\") pod \"node-resolver-kdfqc\" (UID: \"59928e2c-472c-4289-b77d-213f2811bb82\") " pod="openshift-dns/node-resolver-kdfqc" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182053 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-etc-openvswitch\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182056 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-cnibin\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182101 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-hostroot\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182127 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-system-cni-dir\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182144 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9db69d07-9153-4912-af0e-5a73aa0f3f33-cni-binary-copy\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182160 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-tuning-conf-dir\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182178 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xn5jr\" (UniqueName: \"kubernetes.io/projected/109afb78-2acc-4972-9604-32fb10a7ae6b-kube-api-access-xn5jr\") pod \"node-ca-wh4nz\" (UID: \"109afb78-2acc-4972-9604-32fb10a7ae6b\") " pod="openshift-image-registry/node-ca-wh4nz" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182196 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-run-netns\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182233 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-var-lib-kubelet\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182271 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9db69d07-9153-4912-af0e-5a73aa0f3f33-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182294 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-modprobe-d\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182311 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-kubernetes\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182335 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-host\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.182393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182378 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-tmp\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182431 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-os-release\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2gv7\" (UniqueName: \"kubernetes.io/projected/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-kube-api-access-m2gv7\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182506 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/109afb78-2acc-4972-9604-32fb10a7ae6b-serviceca\") pod \"node-ca-wh4nz\" (UID: \"109afb78-2acc-4972-9604-32fb10a7ae6b\") " pod="openshift-image-registry/node-ca-wh4nz" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182547 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-var-lib-openvswitch\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182572 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-run-ovn\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182591 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-var-lib-openvswitch\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182656 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-run-ovn\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182641 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-os-release\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182703 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-conf-dir\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182723 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182734 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-tuned\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182774 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-kubelet\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182793 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87f2496f-9291-4513-812e-23f4132ec75c-konnectivity-ca\") pod \"konnectivity-agent-vhpvh\" (UID: \"87f2496f-9291-4513-812e-23f4132ec75c\") " pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182812 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-run-multus-certs\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182836 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq4z9\" (UniqueName: \"kubernetes.io/projected/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-kube-api-access-mq4z9\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182851 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-sysctl-conf\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.183155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182856 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-kubelet\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182874 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-run\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182902 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7pq5\" (UniqueName: \"kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5\") pod \"network-check-target-zf6t5\" (UID: \"9792d3fc-f6a7-4d56-a906-b650dea53fb2\") " pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182925 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-ovnkube-script-lib\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.182947 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-var-lib-cni-multus\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-cnibin\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183090 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-lib-modules\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183119 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/261d6b5b-fcc0-49a2-a408-af233d788aa3-iptables-alerter-script\") pod \"iptables-alerter-25ptb\" (UID: \"261d6b5b-fcc0-49a2-a408-af233d788aa3\") " pod="openshift-network-operator/iptables-alerter-25ptb" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183148 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-log-socket\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183195 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183223 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-cni-dir\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183256 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-log-socket\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-systemd\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183286 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-sys\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183292 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/109afb78-2acc-4972-9604-32fb10a7ae6b-serviceca\") pod \"node-ca-wh4nz\" (UID: \"109afb78-2acc-4972-9604-32fb10a7ae6b\") " pod="openshift-image-registry/node-ca-wh4nz" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183327 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-ovnkube-config\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183355 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf6q9\" (UniqueName: \"kubernetes.io/projected/a8583dd1-5422-4ab7-be28-3d86c2a2781a-kube-api-access-lf6q9\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.183770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183360 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87f2496f-9291-4513-812e-23f4132ec75c-konnectivity-ca\") pod \"konnectivity-agent-vhpvh\" (UID: \"87f2496f-9291-4513-812e-23f4132ec75c\") " pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183384 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183427 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/261d6b5b-fcc0-49a2-a408-af233d788aa3-host-slash\") pod \"iptables-alerter-25ptb\" (UID: \"261d6b5b-fcc0-49a2-a408-af233d788aa3\") " pod="openshift-network-operator/iptables-alerter-25ptb" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183483 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-run-ovn-kubernetes\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183506 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-ovnkube-script-lib\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183517 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183545 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-var-lib-cni-bin\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183573 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-daemon-config\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183573 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-run-ovn-kubernetes\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183598 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-var-lib-kubelet\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183609 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/261d6b5b-fcc0-49a2-a408-af233d788aa3-host-slash\") pod \"iptables-alerter-25ptb\" (UID: \"261d6b5b-fcc0-49a2-a408-af233d788aa3\") " pod="openshift-network-operator/iptables-alerter-25ptb" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.183576 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183652 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-node-log\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183728 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-node-log\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183681 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-cni-netd\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183774 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-cni-netd\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183861 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/261d6b5b-fcc0-49a2-a408-af233d788aa3-iptables-alerter-script\") pod \"iptables-alerter-25ptb\" (UID: \"261d6b5b-fcc0-49a2-a408-af233d788aa3\") " pod="openshift-network-operator/iptables-alerter-25ptb" Apr 16 18:15:38.184469 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.183943 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs podName:5609e075-d6f5-4642-a711-20f608288876 nodeName:}" failed. No retries permitted until 2026-04-16 18:15:38.683675437 +0000 UTC m=+3.077056677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs") pod "network-metrics-daemon-lp8x6" (UID: "5609e075-d6f5-4642-a711-20f608288876") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.183985 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184018 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-system-cni-dir\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184087 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-run-k8s-cni-cncf-io\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184111 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9db69d07-9153-4912-af0e-5a73aa0f3f33-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184155 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-cni-binary-copy\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-socket-dir-parent\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184210 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-etc-kubernetes\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184237 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-env-overrides\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184270 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-sys-fs\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184303 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59928e2c-472c-4289-b77d-213f2811bb82-hosts-file\") pod \"node-resolver-kdfqc\" (UID: \"59928e2c-472c-4289-b77d-213f2811bb82\") " pod="openshift-dns/node-resolver-kdfqc" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184336 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxzpt\" (UniqueName: \"kubernetes.io/projected/9db69d07-9153-4912-af0e-5a73aa0f3f33-kube-api-access-cxzpt\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184377 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-systemd-units\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184404 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-cni-bin\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184471 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87f2496f-9291-4513-812e-23f4132ec75c-agent-certs\") pod \"konnectivity-agent-vhpvh\" (UID: \"87f2496f-9291-4513-812e-23f4132ec75c\") " pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184515 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-ovnkube-config\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.185128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184528 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-registration-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.185736 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.184556 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-sysconfig\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.185736 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.185181 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-cni-bin\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.185736 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.185297 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.185736 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.185536 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-systemd-units\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.192815 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.186360 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-ovn-node-metrics-cert\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.192815 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.188332 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-env-overrides\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.192815 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.188399 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87f2496f-9291-4513-812e-23f4132ec75c-agent-certs\") pod \"konnectivity-agent-vhpvh\" (UID: \"87f2496f-9291-4513-812e-23f4132ec75c\") " pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:38.192815 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.188457 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 18:15:38.192815 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.188477 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 18:15:38.192815 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.188490 2577 projected.go:194] Error preparing data for projected volume kube-api-access-m7pq5 for pod openshift-network-diagnostics/network-check-target-zf6t5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:38.192815 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.188594 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5 podName:9792d3fc-f6a7-4d56-a906-b650dea53fb2 nodeName:}" failed. No retries permitted until 2026-04-16 18:15:38.688568384 +0000 UTC m=+3.081949633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m7pq5" (UniqueName: "kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5") pod "network-check-target-zf6t5" (UID: "9792d3fc-f6a7-4d56-a906-b650dea53fb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:38.192815 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.190257 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn5jr\" (UniqueName: \"kubernetes.io/projected/109afb78-2acc-4972-9604-32fb10a7ae6b-kube-api-access-xn5jr\") pod \"node-ca-wh4nz\" (UID: \"109afb78-2acc-4972-9604-32fb10a7ae6b\") " pod="openshift-image-registry/node-ca-wh4nz" Apr 16 18:15:38.192815 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.191500 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfc5p\" (UniqueName: \"kubernetes.io/projected/5609e075-d6f5-4642-a711-20f608288876-kube-api-access-vfc5p\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:38.192815 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.191931 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnc2g\" (UniqueName: \"kubernetes.io/projected/e4a39fc7-9f68-446d-8cf6-04b77ee87df2-kube-api-access-qnc2g\") pod \"ovnkube-node-wgfh9\" (UID: \"e4a39fc7-9f68-446d-8cf6-04b77ee87df2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.192815 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.191973 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jrrz\" (UniqueName: \"kubernetes.io/projected/261d6b5b-fcc0-49a2-a408-af233d788aa3-kube-api-access-6jrrz\") pod \"iptables-alerter-25ptb\" (UID: \"261d6b5b-fcc0-49a2-a408-af233d788aa3\") " pod="openshift-network-operator/iptables-alerter-25ptb" Apr 16 18:15:38.265136 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.265103 2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 18:15:38.285924 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.285894 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9db69d07-9153-4912-af0e-5a73aa0f3f33-cni-binary-copy\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.285924 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.285928 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-tuning-conf-dir\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.286114 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.285947 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-run-netns\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286114 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.285966 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-var-lib-kubelet\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286114 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.285990 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9db69d07-9153-4912-af0e-5a73aa0f3f33-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.286114 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286055 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-run-netns\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286114 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286054 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-var-lib-kubelet\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286114 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286091 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-modprobe-d\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286387 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286132 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-kubernetes\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286387 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286163 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-host\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286387 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286187 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-tmp\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286387 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286214 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-os-release\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.286387 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286241 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m2gv7\" (UniqueName: \"kubernetes.io/projected/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-kube-api-access-m2gv7\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286387 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286269 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-os-release\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286387 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286281 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-kubernetes\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286387 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286294 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-conf-dir\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286387 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286317 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-tuned\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286387 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286346 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-run-multus-certs\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286387 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286375 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mq4z9\" (UniqueName: \"kubernetes.io/projected/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-kube-api-access-mq4z9\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286399 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-sysctl-conf\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286421 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-run\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286432 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9db69d07-9153-4912-af0e-5a73aa0f3f33-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286449 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9db69d07-9153-4912-af0e-5a73aa0f3f33-cni-binary-copy\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286467 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-var-lib-cni-multus\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286187 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-modprobe-d\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286242 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-host\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286491 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-cnibin\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286513 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-lib-modules\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286537 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286559 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-cni-dir\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286581 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-systemd\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286704 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-systemd\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286721 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-os-release\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286765 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-tuning-conf-dir\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286776 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-conf-dir\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286786 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-var-lib-cni-multus\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.286908 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286346 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-os-release\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286808 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-sys\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286821 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-cnibin\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286771 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-run-multus-certs\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286936 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-cni-dir\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286936 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lf6q9\" (UniqueName: \"kubernetes.io/projected/a8583dd1-5422-4ab7-be28-3d86c2a2781a-kube-api-access-lf6q9\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286946 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-lib-modules\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287013 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.286990 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287051 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287084 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-var-lib-cni-bin\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287012 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-sys\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287117 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-daemon-config\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287121 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-sysctl-conf\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287148 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-var-lib-kubelet\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287159 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-var-lib-cni-bin\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287187 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-run\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287209 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-var-lib-kubelet\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.287974 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287230 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-system-cni-dir\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287269 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-system-cni-dir\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287280 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-run-k8s-cni-cncf-io\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287320 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9db69d07-9153-4912-af0e-5a73aa0f3f33-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287382 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-cni-binary-copy\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287421 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-host-run-k8s-cni-cncf-io\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287430 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-socket-dir-parent\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287458 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-etc-kubernetes\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287497 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-sys-fs\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287523 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59928e2c-472c-4289-b77d-213f2811bb82-hosts-file\") pod \"node-resolver-kdfqc\" (UID: \"59928e2c-472c-4289-b77d-213f2811bb82\") " pod="openshift-dns/node-resolver-kdfqc" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287550 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cxzpt\" (UniqueName: \"kubernetes.io/projected/9db69d07-9153-4912-af0e-5a73aa0f3f33-kube-api-access-cxzpt\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287581 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-registration-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287604 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-sysconfig\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287655 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d8l4s\" (UniqueName: \"kubernetes.io/projected/59928e2c-472c-4289-b77d-213f2811bb82-kube-api-access-d8l4s\") pod \"node-resolver-kdfqc\" (UID: \"59928e2c-472c-4289-b77d-213f2811bb82\") " pod="openshift-dns/node-resolver-kdfqc" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287679 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-sysctl-d\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287703 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-socket-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287730 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-device-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.288817 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287742 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-daemon-config\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287753 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59928e2c-472c-4289-b77d-213f2811bb82-tmp-dir\") pod \"node-resolver-kdfqc\" (UID: \"59928e2c-472c-4289-b77d-213f2811bb82\") " pod="openshift-dns/node-resolver-kdfqc" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287778 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-cnibin\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287802 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-hostroot\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287816 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-registration-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287828 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-system-cni-dir\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287876 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-multus-socket-dir-parent\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287892 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9db69d07-9153-4912-af0e-5a73aa0f3f33-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287895 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-cni-binary-copy\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287928 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-sys-fs\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287918 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9db69d07-9153-4912-af0e-5a73aa0f3f33-system-cni-dir\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287965 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-device-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287986 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-sysconfig\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.287990 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8583dd1-5422-4ab7-be28-3d86c2a2781a-socket-dir\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.288020 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-etc-kubernetes\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.288130 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-sysctl-d\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.288168 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/59928e2c-472c-4289-b77d-213f2811bb82-hosts-file\") pod \"node-resolver-kdfqc\" (UID: \"59928e2c-472c-4289-b77d-213f2811bb82\") " pod="openshift-dns/node-resolver-kdfqc" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.288186 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-hostroot\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.289400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.288212 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-cnibin\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.290006 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.288231 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59928e2c-472c-4289-b77d-213f2811bb82-tmp-dir\") pod \"node-resolver-kdfqc\" (UID: \"59928e2c-472c-4289-b77d-213f2811bb82\") " pod="openshift-dns/node-resolver-kdfqc" Apr 16 18:15:38.290006 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.289065 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-etc-tuned\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.290006 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.289156 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-tmp\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.296340 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.296125 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2gv7\" (UniqueName: \"kubernetes.io/projected/9f1c0743-c140-4a95-90fb-d4f5a88f3ef6-kube-api-access-m2gv7\") pod \"tuned-9csqh\" (UID: \"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6\") " pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.296340 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.296294 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq4z9\" (UniqueName: \"kubernetes.io/projected/d6e9b61c-b5b2-400a-93ad-c16e7c4989b8-kube-api-access-mq4z9\") pod \"multus-69k7d\" (UID: \"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8\") " pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.296792 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.296698 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8l4s\" (UniqueName: \"kubernetes.io/projected/59928e2c-472c-4289-b77d-213f2811bb82-kube-api-access-d8l4s\") pod \"node-resolver-kdfqc\" (UID: \"59928e2c-472c-4289-b77d-213f2811bb82\") " pod="openshift-dns/node-resolver-kdfqc" Apr 16 18:15:38.296792 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.296767 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxzpt\" (UniqueName: \"kubernetes.io/projected/9db69d07-9153-4912-af0e-5a73aa0f3f33-kube-api-access-cxzpt\") pod \"multus-additional-cni-plugins-59lzl\" (UID: \"9db69d07-9153-4912-af0e-5a73aa0f3f33\") " pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.296869 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.296825 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf6q9\" (UniqueName: \"kubernetes.io/projected/a8583dd1-5422-4ab7-be28-3d86c2a2781a-kube-api-access-lf6q9\") pod \"aws-ebs-csi-driver-node-hdx2q\" (UID: \"a8583dd1-5422-4ab7-be28-3d86c2a2781a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.366797 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.366703 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wh4nz" Apr 16 18:15:38.377642 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.377596 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-25ptb" Apr 16 18:15:38.388430 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.388398 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:15:38.393001 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.392978 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:38.399596 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.399574 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" Apr 16 18:15:38.405680 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.405662 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kdfqc" Apr 16 18:15:38.412259 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.412238 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-69k7d" Apr 16 18:15:38.420945 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.420922 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-59lzl" Apr 16 18:15:38.425639 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.425604 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9csqh" Apr 16 18:15:38.690901 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.690818 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7pq5\" (UniqueName: \"kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5\") pod \"network-check-target-zf6t5\" (UID: \"9792d3fc-f6a7-4d56-a906-b650dea53fb2\") " pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:38.690901 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:38.690875 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:38.691098 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.690986 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:38.691098 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.690993 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 18:15:38.691098 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.691012 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 18:15:38.691098 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.691024 2577 projected.go:194] Error preparing data for projected volume kube-api-access-m7pq5 for pod openshift-network-diagnostics/network-check-target-zf6t5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:38.691098 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.691051 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs podName:5609e075-d6f5-4642-a711-20f608288876 nodeName:}" failed. No retries permitted until 2026-04-16 18:15:39.691033643 +0000 UTC m=+4.084414898 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs") pod "network-metrics-daemon-lp8x6" (UID: "5609e075-d6f5-4642-a711-20f608288876") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:38.691098 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:38.691069 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5 podName:9792d3fc-f6a7-4d56-a906-b650dea53fb2 nodeName:}" failed. No retries permitted until 2026-04-16 18:15:39.691060654 +0000 UTC m=+4.084441895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7pq5" (UniqueName: "kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5") pod "network-check-target-zf6t5" (UID: "9792d3fc-f6a7-4d56-a906-b650dea53fb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:38.795346 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:38.795318 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261d6b5b_fcc0_49a2_a408_af233d788aa3.slice/crio-cbeb834be8b878c81f5656cb6404912e564f18178b4fb509ec75273ae6fd7168 WatchSource:0}: Error finding container cbeb834be8b878c81f5656cb6404912e564f18178b4fb509ec75273ae6fd7168: Status 404 returned error can't find the container with id cbeb834be8b878c81f5656cb6404912e564f18178b4fb509ec75273ae6fd7168 Apr 16 18:15:38.796618 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:38.796544 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109afb78_2acc_4972_9604_32fb10a7ae6b.slice/crio-c995658c0a6c9b29c2ccd4a93ebc6d4a3d0673550d68135a0ec987825d33075f WatchSource:0}: Error finding container c995658c0a6c9b29c2ccd4a93ebc6d4a3d0673550d68135a0ec987825d33075f: Status 404 returned error can't find the container with id c995658c0a6c9b29c2ccd4a93ebc6d4a3d0673550d68135a0ec987825d33075f Apr 16 18:15:38.798787 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:38.798751 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87f2496f_9291_4513_812e_23f4132ec75c.slice/crio-eb09b8a16ad99639524a14ef0e8836f535d0b1c03ded0b3cc1daf5f700fd7879 WatchSource:0}: Error finding container eb09b8a16ad99639524a14ef0e8836f535d0b1c03ded0b3cc1daf5f700fd7879: Status 404 returned error can't find the container with id eb09b8a16ad99639524a14ef0e8836f535d0b1c03ded0b3cc1daf5f700fd7879 Apr 16 18:15:38.799506 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:38.799481 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f1c0743_c140_4a95_90fb_d4f5a88f3ef6.slice/crio-abc7c100c922deb7ae5fd9e74781573ba5148122e79583fca0a607890aa4b525 WatchSource:0}: Error finding container abc7c100c922deb7ae5fd9e74781573ba5148122e79583fca0a607890aa4b525: Status 404 returned error can't find the container with id abc7c100c922deb7ae5fd9e74781573ba5148122e79583fca0a607890aa4b525 Apr 16 18:15:38.801320 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:38.801303 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59928e2c_472c_4289_b77d_213f2811bb82.slice/crio-c3ac787c2fbd62d193e21ae3ba8d935bb889334d4bd89648b4029bd6829ab401 WatchSource:0}: Error finding container c3ac787c2fbd62d193e21ae3ba8d935bb889334d4bd89648b4029bd6829ab401: Status 404 returned error can't find the container with id c3ac787c2fbd62d193e21ae3ba8d935bb889334d4bd89648b4029bd6829ab401 Apr 16 18:15:38.823906 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:38.823873 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e9b61c_b5b2_400a_93ad_c16e7c4989b8.slice/crio-39e8f9b769b6c9f451196d5a4af54f1849bcd02f5b69bac36046c5d545ec298b WatchSource:0}: Error finding container 39e8f9b769b6c9f451196d5a4af54f1849bcd02f5b69bac36046c5d545ec298b: Status 404 returned error can't find the container with id 39e8f9b769b6c9f451196d5a4af54f1849bcd02f5b69bac36046c5d545ec298b Apr 16 18:15:38.824671 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:38.824651 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8583dd1_5422_4ab7_be28_3d86c2a2781a.slice/crio-9982cf938649536e168fa4d52f537d27bd86f7abaf78976003cc28f8627f92f8 WatchSource:0}: Error finding container 9982cf938649536e168fa4d52f537d27bd86f7abaf78976003cc28f8627f92f8: Status 404 returned error can't find the container with id 9982cf938649536e168fa4d52f537d27bd86f7abaf78976003cc28f8627f92f8 Apr 16 18:15:38.825551 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:38.825528 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4a39fc7_9f68_446d_8cf6_04b77ee87df2.slice/crio-89d22f8b7b8b328866fe2ea4014520dd05d41b3a09c670ee0b3867b09b773ef9 WatchSource:0}: Error finding container 89d22f8b7b8b328866fe2ea4014520dd05d41b3a09c670ee0b3867b09b773ef9: Status 404 returned error can't find the container with id 89d22f8b7b8b328866fe2ea4014520dd05d41b3a09c670ee0b3867b09b773ef9 Apr 16 18:15:38.826331 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:15:38.826310 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9db69d07_9153_4912_af0e_5a73aa0f3f33.slice/crio-966009884f6fd151f75ce63cc09babcfe63353d98108c7174d92dec0a28e711b WatchSource:0}: Error finding container 966009884f6fd151f75ce63cc09babcfe63353d98108c7174d92dec0a28e711b: Status 404 returned error can't find the container with id 966009884f6fd151f75ce63cc09babcfe63353d98108c7174d92dec0a28e711b Apr 16 18:15:39.116095 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.115796 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-15 18:10:37 +0000 UTC" deadline="2027-10-04 20:10:55.003321168 +0000 UTC" Apr 16 18:15:39.116095 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.116037 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12865h55m15.887290509s" Apr 16 18:15:39.209606 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.209575 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:39.209782 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:39.209722 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:39.230741 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.230702 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59lzl" event={"ID":"9db69d07-9153-4912-af0e-5a73aa0f3f33","Type":"ContainerStarted","Data":"966009884f6fd151f75ce63cc09babcfe63353d98108c7174d92dec0a28e711b"} Apr 16 18:15:39.232178 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.232148 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-69k7d" event={"ID":"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8","Type":"ContainerStarted","Data":"39e8f9b769b6c9f451196d5a4af54f1849bcd02f5b69bac36046c5d545ec298b"} Apr 16 18:15:39.234743 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.234714 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kdfqc" event={"ID":"59928e2c-472c-4289-b77d-213f2811bb82","Type":"ContainerStarted","Data":"c3ac787c2fbd62d193e21ae3ba8d935bb889334d4bd89648b4029bd6829ab401"} Apr 16 18:15:39.244848 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.244807 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vhpvh" event={"ID":"87f2496f-9291-4513-812e-23f4132ec75c","Type":"ContainerStarted","Data":"eb09b8a16ad99639524a14ef0e8836f535d0b1c03ded0b3cc1daf5f700fd7879"} Apr 16 18:15:39.250688 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.249880 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-22.ec2.internal" event={"ID":"9651c74f7f561de9ee59b0475859f99e","Type":"ContainerStarted","Data":"5ba20c876cfff6527300a307f0ea335318b7babec62de3c62c1a8dc0edd4dcf6"} Apr 16 18:15:39.260783 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.260615 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" event={"ID":"a8583dd1-5422-4ab7-be28-3d86c2a2781a","Type":"ContainerStarted","Data":"9982cf938649536e168fa4d52f537d27bd86f7abaf78976003cc28f8627f92f8"} Apr 16 18:15:39.267977 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.267899 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" event={"ID":"e4a39fc7-9f68-446d-8cf6-04b77ee87df2","Type":"ContainerStarted","Data":"89d22f8b7b8b328866fe2ea4014520dd05d41b3a09c670ee0b3867b09b773ef9"} Apr 16 18:15:39.273090 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.272847 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9csqh" event={"ID":"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6","Type":"ContainerStarted","Data":"abc7c100c922deb7ae5fd9e74781573ba5148122e79583fca0a607890aa4b525"} Apr 16 18:15:39.280154 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.280118 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wh4nz" event={"ID":"109afb78-2acc-4972-9604-32fb10a7ae6b","Type":"ContainerStarted","Data":"c995658c0a6c9b29c2ccd4a93ebc6d4a3d0673550d68135a0ec987825d33075f"} Apr 16 18:15:39.289086 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.289049 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-25ptb" event={"ID":"261d6b5b-fcc0-49a2-a408-af233d788aa3","Type":"ContainerStarted","Data":"cbeb834be8b878c81f5656cb6404912e564f18178b4fb509ec75273ae6fd7168"} Apr 16 18:15:39.697248 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.697211 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7pq5\" (UniqueName: \"kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5\") pod \"network-check-target-zf6t5\" (UID: \"9792d3fc-f6a7-4d56-a906-b650dea53fb2\") " pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:39.697436 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:39.697272 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:39.697436 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:39.697421 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:39.697548 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:39.697483 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs podName:5609e075-d6f5-4642-a711-20f608288876 nodeName:}" failed. No retries permitted until 2026-04-16 18:15:41.697464848 +0000 UTC m=+6.090846098 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs") pod "network-metrics-daemon-lp8x6" (UID: "5609e075-d6f5-4642-a711-20f608288876") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:39.697922 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:39.697902 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 18:15:39.698015 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:39.697927 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 18:15:39.698015 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:39.697940 2577 projected.go:194] Error preparing data for projected volume kube-api-access-m7pq5 for pod openshift-network-diagnostics/network-check-target-zf6t5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:39.698015 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:39.697986 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5 podName:9792d3fc-f6a7-4d56-a906-b650dea53fb2 nodeName:}" failed. No retries permitted until 2026-04-16 18:15:41.697970424 +0000 UTC m=+6.091351676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7pq5" (UniqueName: "kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5") pod "network-check-target-zf6t5" (UID: "9792d3fc-f6a7-4d56-a906-b650dea53fb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:40.210907 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:40.210825 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:40.211381 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:40.210979 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:15:40.305746 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:40.305712 2577 generic.go:358] "Generic (PLEG): container finished" podID="d3c5a36ec21f8e94d847668d8289fd26" containerID="9ad66927a6576505c9111fe18fe1d58c581a85bff1d277d446b0640b1eaea671" exitCode=0 Apr 16 18:15:40.306578 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:40.306547 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" event={"ID":"d3c5a36ec21f8e94d847668d8289fd26","Type":"ContainerDied","Data":"9ad66927a6576505c9111fe18fe1d58c581a85bff1d277d446b0640b1eaea671"} Apr 16 18:15:40.325120 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:40.325059 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-22.ec2.internal" podStartSLOduration=3.325039688 podStartE2EDuration="3.325039688s" podCreationTimestamp="2026-04-16 18:15:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 18:15:39.264989177 +0000 UTC m=+3.658370434" watchObservedRunningTime="2026-04-16 18:15:40.325039688 +0000 UTC m=+4.718420949" Apr 16 18:15:41.209852 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:41.209815 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:41.210047 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:41.209950 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:41.318891 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:41.318853 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" event={"ID":"d3c5a36ec21f8e94d847668d8289fd26","Type":"ContainerStarted","Data":"7773ffc3c644f131ca9a165fd53c1ce3f99414facfd9496c09953d32bcff981f"} Apr 16 18:15:41.715071 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:41.715031 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:41.715259 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:41.715121 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7pq5\" (UniqueName: \"kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5\") pod \"network-check-target-zf6t5\" (UID: \"9792d3fc-f6a7-4d56-a906-b650dea53fb2\") " pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:41.715312 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:41.715259 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 18:15:41.715312 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:41.715277 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 18:15:41.715312 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:41.715289 2577 projected.go:194] Error preparing data for projected volume kube-api-access-m7pq5 for pod openshift-network-diagnostics/network-check-target-zf6t5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:41.715458 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:41.715347 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5 podName:9792d3fc-f6a7-4d56-a906-b650dea53fb2 nodeName:}" failed. No retries permitted until 2026-04-16 18:15:45.715329209 +0000 UTC m=+10.108710447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7pq5" (UniqueName: "kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5") pod "network-check-target-zf6t5" (UID: "9792d3fc-f6a7-4d56-a906-b650dea53fb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:41.715888 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:41.715767 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:41.715888 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:41.715823 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs podName:5609e075-d6f5-4642-a711-20f608288876 nodeName:}" failed. No retries permitted until 2026-04-16 18:15:45.715806398 +0000 UTC m=+10.109187634 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs") pod "network-metrics-daemon-lp8x6" (UID: "5609e075-d6f5-4642-a711-20f608288876") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:42.212882 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:42.212761 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:42.213036 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:42.212944 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:15:43.209500 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:43.209447 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:43.209983 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:43.209571 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:44.210926 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:44.210430 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:44.210926 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:44.210557 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:15:45.210163 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:45.210125 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:45.210319 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:45.210238 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:45.750702 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:45.750658 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7pq5\" (UniqueName: \"kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5\") pod \"network-check-target-zf6t5\" (UID: \"9792d3fc-f6a7-4d56-a906-b650dea53fb2\") " pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:45.751190 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:45.750725 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:45.751190 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:45.750868 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:45.751190 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:45.750919 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs podName:5609e075-d6f5-4642-a711-20f608288876 nodeName:}" failed. No retries permitted until 2026-04-16 18:15:53.750904825 +0000 UTC m=+18.144286074 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs") pod "network-metrics-daemon-lp8x6" (UID: "5609e075-d6f5-4642-a711-20f608288876") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:45.751363 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:45.751289 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 18:15:45.751363 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:45.751307 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 18:15:45.751363 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:45.751320 2577 projected.go:194] Error preparing data for projected volume kube-api-access-m7pq5 for pod openshift-network-diagnostics/network-check-target-zf6t5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:45.751520 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:45.751368 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5 podName:9792d3fc-f6a7-4d56-a906-b650dea53fb2 nodeName:}" failed. No retries permitted until 2026-04-16 18:15:53.751355267 +0000 UTC m=+18.144736501 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7pq5" (UniqueName: "kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5") pod "network-check-target-zf6t5" (UID: "9792d3fc-f6a7-4d56-a906-b650dea53fb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:46.218206 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:46.218125 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:46.218505 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:46.218467 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:15:47.209880 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:47.209840 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:47.210331 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:47.209973 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:48.210965 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:48.210426 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:48.210965 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:48.210556 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:15:49.209794 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:49.209761 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:49.209975 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:49.209880 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:50.210142 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:50.210109 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:50.210575 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:50.210248 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:15:51.210057 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:51.210022 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:51.210240 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:51.210131 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:52.210384 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:52.210344 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:52.210871 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:52.210490 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:15:53.210035 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:53.210000 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:53.210217 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:53.210116 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:53.809834 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:53.809783 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7pq5\" (UniqueName: \"kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5\") pod \"network-check-target-zf6t5\" (UID: \"9792d3fc-f6a7-4d56-a906-b650dea53fb2\") " pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:53.810312 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:53.809851 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:53.810312 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:53.809991 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:53.810312 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:53.810007 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 18:15:53.810312 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:53.810034 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 18:15:53.810312 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:53.810047 2577 projected.go:194] Error preparing data for projected volume kube-api-access-m7pq5 for pod openshift-network-diagnostics/network-check-target-zf6t5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:53.810312 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:53.810059 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs podName:5609e075-d6f5-4642-a711-20f608288876 nodeName:}" failed. No retries permitted until 2026-04-16 18:16:09.810038904 +0000 UTC m=+34.203420141 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs") pod "network-metrics-daemon-lp8x6" (UID: "5609e075-d6f5-4642-a711-20f608288876") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:15:53.810312 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:53.810098 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5 podName:9792d3fc-f6a7-4d56-a906-b650dea53fb2 nodeName:}" failed. No retries permitted until 2026-04-16 18:16:09.810085448 +0000 UTC m=+34.203466687 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7pq5" (UniqueName: "kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5") pod "network-check-target-zf6t5" (UID: "9792d3fc-f6a7-4d56-a906-b650dea53fb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:15:54.209519 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:54.209424 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:54.209755 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:54.209560 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:15:55.209739 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:55.209696 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:55.210209 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:55.209824 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:56.212096 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:56.212068 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:56.212439 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:56.212159 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:15:57.209944 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.209644 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:57.210118 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:57.210018 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:57.349165 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.349127 2577 generic.go:358] "Generic (PLEG): container finished" podID="9db69d07-9153-4912-af0e-5a73aa0f3f33" containerID="66cab7168f32ef24a89a87c3a788b700c5ef05cea07bbae03cced1bbc0a5405e" exitCode=0 Apr 16 18:15:57.350019 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.349212 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59lzl" event={"ID":"9db69d07-9153-4912-af0e-5a73aa0f3f33","Type":"ContainerDied","Data":"66cab7168f32ef24a89a87c3a788b700c5ef05cea07bbae03cced1bbc0a5405e"} Apr 16 18:15:57.351121 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.351086 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-69k7d" event={"ID":"d6e9b61c-b5b2-400a-93ad-c16e7c4989b8","Type":"ContainerStarted","Data":"4ec557a6f80090107006740dfd2440557729184045e123048045edb35aa86222"} Apr 16 18:15:57.353436 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.353412 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kdfqc" event={"ID":"59928e2c-472c-4289-b77d-213f2811bb82","Type":"ContainerStarted","Data":"a2c77cdfae49a39cf97a439055482c5fb977f348f39ea0bc9efa878e74e48b01"} Apr 16 18:15:57.354928 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.354900 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vhpvh" event={"ID":"87f2496f-9291-4513-812e-23f4132ec75c","Type":"ContainerStarted","Data":"15097122a85faf6c7bc7d53bac902b75b2ce091f055497514a801bbccbfae2d0"} Apr 16 18:15:57.356465 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.356435 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" event={"ID":"a8583dd1-5422-4ab7-be28-3d86c2a2781a","Type":"ContainerStarted","Data":"0adce9d4c422bd0a9905ff9dadf52808beb7b169eef1775fd64ed9f105e9f18f"} Apr 16 18:15:57.359680 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.359606 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" event={"ID":"e4a39fc7-9f68-446d-8cf6-04b77ee87df2","Type":"ContainerStarted","Data":"b71e16e1f3f283b62d8af1c1fcd3bcd648aa7db661a17022a6d38aea8cccd2ed"} Apr 16 18:15:57.359680 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.359661 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" event={"ID":"e4a39fc7-9f68-446d-8cf6-04b77ee87df2","Type":"ContainerStarted","Data":"42eac38cd0d3582d3ab0b24c820765483042da9089cc1d74fde4eb8dea8bc7c3"} Apr 16 18:15:57.359809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.359691 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" event={"ID":"e4a39fc7-9f68-446d-8cf6-04b77ee87df2","Type":"ContainerStarted","Data":"91b54dcbd3d55d04fdfac3081b78b3ad991052e918f455f15aecf97ad5783f90"} Apr 16 18:15:57.359809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.359703 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" event={"ID":"e4a39fc7-9f68-446d-8cf6-04b77ee87df2","Type":"ContainerStarted","Data":"e6202cd2cc163b86acb293f1b9c7c1a4aa361b2b1ea0169d2c8855a495303c63"} Apr 16 18:15:57.359809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.359716 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" event={"ID":"e4a39fc7-9f68-446d-8cf6-04b77ee87df2","Type":"ContainerStarted","Data":"a3156da27d350b78d8b545dfed3995f37682bb4c5107ea610ef2adf8b72c9ff1"} Apr 16 18:15:57.359809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.359728 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" event={"ID":"e4a39fc7-9f68-446d-8cf6-04b77ee87df2","Type":"ContainerStarted","Data":"32b461f3986c6638bc86a84c39236c1c2b2dc7ee2b2f4d9b3c2f0af7caf784e7"} Apr 16 18:15:57.361179 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.361147 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9csqh" event={"ID":"9f1c0743-c140-4a95-90fb-d4f5a88f3ef6","Type":"ContainerStarted","Data":"eb4297a2ccbc8ffdb0c5b6cbb854006c7202eb96eebcce4de225bcdfba9960f8"} Apr 16 18:15:57.362531 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.362509 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wh4nz" event={"ID":"109afb78-2acc-4972-9604-32fb10a7ae6b","Type":"ContainerStarted","Data":"d61ea1344b100d24051c9884a6609a1113b3407144091f6ff329d73afe33a827"} Apr 16 18:15:57.370057 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.370015 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-22.ec2.internal" podStartSLOduration=20.370003698 podStartE2EDuration="20.370003698s" podCreationTimestamp="2026-04-16 18:15:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 18:15:41.334885426 +0000 UTC m=+5.728266684" watchObservedRunningTime="2026-04-16 18:15:57.370003698 +0000 UTC m=+21.763384955" Apr 16 18:15:57.384975 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.384924 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9csqh" podStartSLOduration=3.954026114 podStartE2EDuration="21.384911101s" podCreationTimestamp="2026-04-16 18:15:36 +0000 UTC" firstStartedPulling="2026-04-16 18:15:38.822453805 +0000 UTC m=+3.215835039" lastFinishedPulling="2026-04-16 18:15:56.253338776 +0000 UTC m=+20.646720026" observedRunningTime="2026-04-16 18:15:57.384756974 +0000 UTC m=+21.778138222" watchObservedRunningTime="2026-04-16 18:15:57.384911101 +0000 UTC m=+21.778292359" Apr 16 18:15:57.399206 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.399145 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vhpvh" podStartSLOduration=3.96822146 podStartE2EDuration="21.39912754s" podCreationTimestamp="2026-04-16 18:15:36 +0000 UTC" firstStartedPulling="2026-04-16 18:15:38.822494062 +0000 UTC m=+3.215875296" lastFinishedPulling="2026-04-16 18:15:56.253400138 +0000 UTC m=+20.646781376" observedRunningTime="2026-04-16 18:15:57.398415681 +0000 UTC m=+21.791796938" watchObservedRunningTime="2026-04-16 18:15:57.39912754 +0000 UTC m=+21.792508797" Apr 16 18:15:57.427486 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.427434 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-kdfqc" podStartSLOduration=3.996573386 podStartE2EDuration="21.427419057s" podCreationTimestamp="2026-04-16 18:15:36 +0000 UTC" firstStartedPulling="2026-04-16 18:15:38.822494905 +0000 UTC m=+3.215876139" lastFinishedPulling="2026-04-16 18:15:56.253340574 +0000 UTC m=+20.646721810" observedRunningTime="2026-04-16 18:15:57.427086289 +0000 UTC m=+21.820467545" watchObservedRunningTime="2026-04-16 18:15:57.427419057 +0000 UTC m=+21.820800313" Apr 16 18:15:57.427644 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.427509 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wh4nz" podStartSLOduration=8.695203331 podStartE2EDuration="21.427505338s" podCreationTimestamp="2026-04-16 18:15:36 +0000 UTC" firstStartedPulling="2026-04-16 18:15:38.798583885 +0000 UTC m=+3.191965122" lastFinishedPulling="2026-04-16 18:15:51.530885891 +0000 UTC m=+15.924267129" observedRunningTime="2026-04-16 18:15:57.412963466 +0000 UTC m=+21.806344714" watchObservedRunningTime="2026-04-16 18:15:57.427505338 +0000 UTC m=+21.820886605" Apr 16 18:15:57.447837 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.447773 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-69k7d" podStartSLOduration=3.98626741 podStartE2EDuration="21.447756789s" podCreationTimestamp="2026-04-16 18:15:36 +0000 UTC" firstStartedPulling="2026-04-16 18:15:38.829575019 +0000 UTC m=+3.222956258" lastFinishedPulling="2026-04-16 18:15:56.291064382 +0000 UTC m=+20.684445637" observedRunningTime="2026-04-16 18:15:57.447524434 +0000 UTC m=+21.840905691" watchObservedRunningTime="2026-04-16 18:15:57.447756789 +0000 UTC m=+21.841138046" Apr 16 18:15:57.589075 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:57.589047 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Apr 16 18:15:58.150864 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:58.150738 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-04-16T18:15:57.589069255Z","UUID":"c3d9caca-47c2-40d9-9944-55d629605d76","Handler":null,"Name":"","Endpoint":""} Apr 16 18:15:58.154432 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:58.154405 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Apr 16 18:15:58.154588 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:58.154443 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Apr 16 18:15:58.213080 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:58.213048 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:15:58.213250 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:58.213183 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:15:58.367085 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:58.367050 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" event={"ID":"a8583dd1-5422-4ab7-be28-3d86c2a2781a","Type":"ContainerStarted","Data":"888eb1a962eb6d5290d0de764403381ed10f3ddb7354a479cd14a5592627a3cb"} Apr 16 18:15:58.368818 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:58.368790 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-25ptb" event={"ID":"261d6b5b-fcc0-49a2-a408-af233d788aa3","Type":"ContainerStarted","Data":"55012d7840fd3844e0ce5b661877fbbc6c678a0775c21325f151fa87d85015ff"} Apr 16 18:15:59.210298 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:59.210272 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:15:59.210431 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:15:59.210398 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:15:59.289820 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:59.289798 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:59.290465 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:59.290442 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:59.305474 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:59.305418 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-25ptb" podStartSLOduration=5.849127825 podStartE2EDuration="23.305398274s" podCreationTimestamp="2026-04-16 18:15:36 +0000 UTC" firstStartedPulling="2026-04-16 18:15:38.797786429 +0000 UTC m=+3.191167678" lastFinishedPulling="2026-04-16 18:15:56.254056893 +0000 UTC m=+20.647438127" observedRunningTime="2026-04-16 18:15:58.385769529 +0000 UTC m=+22.779150786" watchObservedRunningTime="2026-04-16 18:15:59.305398274 +0000 UTC m=+23.698779532" Apr 16 18:15:59.372566 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:59.372529 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" event={"ID":"a8583dd1-5422-4ab7-be28-3d86c2a2781a","Type":"ContainerStarted","Data":"09545a96e4426d6471cdd3d59b612fc38af48eb9405efc5f7a7df93f0efa6b8b"} Apr 16 18:15:59.376017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:59.375964 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" event={"ID":"e4a39fc7-9f68-446d-8cf6-04b77ee87df2","Type":"ContainerStarted","Data":"025e2e4872005f6ff7b4a5b72bc4e6684090f24d60c9c4c0b83a8747f697d23e"} Apr 16 18:15:59.376442 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:59.376319 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:59.376722 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:59.376705 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vhpvh" Apr 16 18:15:59.392322 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:15:59.392263 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdx2q" podStartSLOduration=3.569197709 podStartE2EDuration="23.392246292s" podCreationTimestamp="2026-04-16 18:15:36 +0000 UTC" firstStartedPulling="2026-04-16 18:15:38.829566992 +0000 UTC m=+3.222948231" lastFinishedPulling="2026-04-16 18:15:58.652615564 +0000 UTC m=+23.045996814" observedRunningTime="2026-04-16 18:15:59.391814279 +0000 UTC m=+23.785195537" watchObservedRunningTime="2026-04-16 18:15:59.392246292 +0000 UTC m=+23.785627550" Apr 16 18:16:00.209916 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:00.209884 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:16:00.210086 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:00.210000 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:16:01.210213 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:01.210167 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:01.210763 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:01.210299 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:16:02.210116 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:02.209883 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:16:02.210288 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:02.210236 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:16:02.383978 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:02.383945 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" event={"ID":"e4a39fc7-9f68-446d-8cf6-04b77ee87df2","Type":"ContainerStarted","Data":"19b326aa360362299a32c2fa4a832aa7e71376ae17e56fb1a1848054c8ddbc5c"} Apr 16 18:16:02.384390 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:02.384345 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:16:02.385704 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:02.385679 2577 generic.go:358] "Generic (PLEG): container finished" podID="9db69d07-9153-4912-af0e-5a73aa0f3f33" containerID="6a111b2ee0ed11cacbd43b801bb6c63908cc8f63f4929f954594ccfcbd786191" exitCode=0 Apr 16 18:16:02.385803 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:02.385714 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59lzl" event={"ID":"9db69d07-9153-4912-af0e-5a73aa0f3f33","Type":"ContainerDied","Data":"6a111b2ee0ed11cacbd43b801bb6c63908cc8f63f4929f954594ccfcbd786191"} Apr 16 18:16:02.399687 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:02.399660 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:16:02.408715 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:02.408669 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" podStartSLOduration=8.893714564 podStartE2EDuration="26.408657309s" podCreationTimestamp="2026-04-16 18:15:36 +0000 UTC" firstStartedPulling="2026-04-16 18:15:38.829553739 +0000 UTC m=+3.222934982" lastFinishedPulling="2026-04-16 18:15:56.344496478 +0000 UTC m=+20.737877727" observedRunningTime="2026-04-16 18:16:02.408375789 +0000 UTC m=+26.801757046" watchObservedRunningTime="2026-04-16 18:16:02.408657309 +0000 UTC m=+26.802038566" Apr 16 18:16:03.209542 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:03.209380 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:03.209712 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:03.209603 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:16:03.390295 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:03.390186 2577 generic.go:358] "Generic (PLEG): container finished" podID="9db69d07-9153-4912-af0e-5a73aa0f3f33" containerID="0e5cc52dcdbdeda33323fc4b7c07cf59cba063a9dd7144292dc15abf9233d3f8" exitCode=0 Apr 16 18:16:03.390295 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:03.390272 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59lzl" event={"ID":"9db69d07-9153-4912-af0e-5a73aa0f3f33","Type":"ContainerDied","Data":"0e5cc52dcdbdeda33323fc4b7c07cf59cba063a9dd7144292dc15abf9233d3f8"} Apr 16 18:16:03.390909 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:03.390875 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:16:03.390909 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:03.390899 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:16:03.395937 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:03.395802 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zf6t5"] Apr 16 18:16:03.395937 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:03.395900 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:03.396123 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:03.395985 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:16:03.396549 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:03.396528 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lp8x6"] Apr 16 18:16:03.396679 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:03.396647 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:16:03.396747 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:03.396729 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:16:03.408349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:03.408299 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:16:04.394797 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:04.394708 2577 generic.go:358] "Generic (PLEG): container finished" podID="9db69d07-9153-4912-af0e-5a73aa0f3f33" containerID="4196888637fedde9ce6c42504e383e96e01f50d856f8e041c1e7f62a8e0a902c" exitCode=0 Apr 16 18:16:04.395158 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:04.394798 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59lzl" event={"ID":"9db69d07-9153-4912-af0e-5a73aa0f3f33","Type":"ContainerDied","Data":"4196888637fedde9ce6c42504e383e96e01f50d856f8e041c1e7f62a8e0a902c"} Apr 16 18:16:05.210358 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:05.210325 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:05.210537 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:05.210366 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:16:05.210537 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:05.210440 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:16:05.210667 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:05.210597 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:16:07.210268 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:07.210228 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:16:07.210805 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:07.210228 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:07.210805 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:07.210364 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:16:07.210805 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:07.210429 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:16:09.209566 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:09.209528 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:09.210153 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:09.209534 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:16:09.210153 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:09.209693 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zf6t5" podUID="9792d3fc-f6a7-4d56-a906-b650dea53fb2" Apr 16 18:16:09.210153 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:09.209790 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:16:09.828961 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:09.828930 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7pq5\" (UniqueName: \"kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5\") pod \"network-check-target-zf6t5\" (UID: \"9792d3fc-f6a7-4d56-a906-b650dea53fb2\") " pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:09.828961 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:09.828967 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:16:09.829231 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:09.829055 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:16:09.829231 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:09.829078 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 18:16:09.829231 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:09.829096 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 18:16:09.829231 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:09.829105 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs podName:5609e075-d6f5-4642-a711-20f608288876 nodeName:}" failed. No retries permitted until 2026-04-16 18:16:41.829091738 +0000 UTC m=+66.222472973 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs") pod "network-metrics-daemon-lp8x6" (UID: "5609e075-d6f5-4642-a711-20f608288876") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 18:16:09.829231 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:09.829105 2577 projected.go:194] Error preparing data for projected volume kube-api-access-m7pq5 for pod openshift-network-diagnostics/network-check-target-zf6t5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:16:09.829231 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:09.829147 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5 podName:9792d3fc-f6a7-4d56-a906-b650dea53fb2 nodeName:}" failed. No retries permitted until 2026-04-16 18:16:41.829135609 +0000 UTC m=+66.222516844 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7pq5" (UniqueName: "kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5") pod "network-check-target-zf6t5" (UID: "9792d3fc-f6a7-4d56-a906-b650dea53fb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 18:16:10.408298 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.408014 2577 generic.go:358] "Generic (PLEG): container finished" podID="9db69d07-9153-4912-af0e-5a73aa0f3f33" containerID="885c8ec24ec229f556238da3cc2ce7375856ec4886bcfe9a1bf60cefd32bb597" exitCode=0 Apr 16 18:16:10.408298 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.408090 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59lzl" event={"ID":"9db69d07-9153-4912-af0e-5a73aa0f3f33","Type":"ContainerDied","Data":"885c8ec24ec229f556238da3cc2ce7375856ec4886bcfe9a1bf60cefd32bb597"} Apr 16 18:16:10.434374 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.434348 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-22.ec2.internal" event="NodeReady" Apr 16 18:16:10.434524 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.434458 2577 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Apr 16 18:16:10.486559 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.485853 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7mtp7"] Apr 16 18:16:10.502868 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.502830 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fqnj7"] Apr 16 18:16:10.513263 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.513229 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7mtp7"] Apr 16 18:16:10.513400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.513269 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fqnj7"] Apr 16 18:16:10.513400 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.513374 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.513485 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.513431 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:10.517277 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.517251 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Apr 16 18:16:10.517277 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.517258 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Apr 16 18:16:10.517477 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.517282 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Apr 16 18:16:10.517477 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.517309 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4scgb\"" Apr 16 18:16:10.517477 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.517353 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Apr 16 18:16:10.517739 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.517725 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Apr 16 18:16:10.517835 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.517820 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fxg25\"" Apr 16 18:16:10.634043 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.634011 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-config-volume\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.634043 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.634046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-tmp-dir\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.634245 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.634067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9njl5\" (UniqueName: \"kubernetes.io/projected/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-kube-api-access-9njl5\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.634245 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.634165 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmgbs\" (UniqueName: \"kubernetes.io/projected/1f3ce71d-67a7-431e-9a65-37dd55851b2c-kube-api-access-xmgbs\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:10.634245 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.634204 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:10.634245 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.634225 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.735391 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.735300 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xmgbs\" (UniqueName: \"kubernetes.io/projected/1f3ce71d-67a7-431e-9a65-37dd55851b2c-kube-api-access-xmgbs\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:10.735391 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.735347 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:10.735391 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.735371 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.735686 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.735412 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-config-volume\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.735686 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:10.735469 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 18:16:10.735686 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:10.735493 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 18:16:10.735686 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.735534 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-tmp-dir\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.735686 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:10.735555 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls podName:e7a02fec-362b-4be1-bf77-0fbf2a0a986e nodeName:}" failed. No retries permitted until 2026-04-16 18:16:11.235531178 +0000 UTC m=+35.628912439 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls") pod "dns-default-7mtp7" (UID: "e7a02fec-362b-4be1-bf77-0fbf2a0a986e") : secret "dns-default-metrics-tls" not found Apr 16 18:16:10.735686 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:10.735579 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert podName:1f3ce71d-67a7-431e-9a65-37dd55851b2c nodeName:}" failed. No retries permitted until 2026-04-16 18:16:11.23556954 +0000 UTC m=+35.628950775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert") pod "ingress-canary-fqnj7" (UID: "1f3ce71d-67a7-431e-9a65-37dd55851b2c") : secret "canary-serving-cert" not found Apr 16 18:16:10.735686 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.735615 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9njl5\" (UniqueName: \"kubernetes.io/projected/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-kube-api-access-9njl5\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.735925 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.735897 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-tmp-dir\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.735984 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.735968 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-config-volume\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.748229 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.748198 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9njl5\" (UniqueName: \"kubernetes.io/projected/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-kube-api-access-9njl5\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:10.748405 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:10.748383 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmgbs\" (UniqueName: \"kubernetes.io/projected/1f3ce71d-67a7-431e-9a65-37dd55851b2c-kube-api-access-xmgbs\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:11.209949 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:11.209912 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:16:11.210125 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:11.209912 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:11.212372 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:11.212350 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4plx6\"" Apr 16 18:16:11.212372 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:11.212363 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 16 18:16:11.212547 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:11.212368 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 16 18:16:11.212547 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:11.212370 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 16 18:16:11.212547 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:11.212502 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xl8bl\"" Apr 16 18:16:11.239071 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:11.239038 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:11.239071 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:11.239077 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:11.239323 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:11.239193 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 18:16:11.239323 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:11.239198 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 18:16:11.239323 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:11.239244 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls podName:e7a02fec-362b-4be1-bf77-0fbf2a0a986e nodeName:}" failed. No retries permitted until 2026-04-16 18:16:12.239231032 +0000 UTC m=+36.632612267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls") pod "dns-default-7mtp7" (UID: "e7a02fec-362b-4be1-bf77-0fbf2a0a986e") : secret "dns-default-metrics-tls" not found Apr 16 18:16:11.239323 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:11.239269 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert podName:1f3ce71d-67a7-431e-9a65-37dd55851b2c nodeName:}" failed. No retries permitted until 2026-04-16 18:16:12.239250464 +0000 UTC m=+36.632631714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert") pod "ingress-canary-fqnj7" (UID: "1f3ce71d-67a7-431e-9a65-37dd55851b2c") : secret "canary-serving-cert" not found Apr 16 18:16:11.412903 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:11.412861 2577 generic.go:358] "Generic (PLEG): container finished" podID="9db69d07-9153-4912-af0e-5a73aa0f3f33" containerID="1b820ff296876a0d3137a221f2bc85354971678f015e5e61756064bd7cdc10d5" exitCode=0 Apr 16 18:16:11.413272 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:11.412911 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59lzl" event={"ID":"9db69d07-9153-4912-af0e-5a73aa0f3f33","Type":"ContainerDied","Data":"1b820ff296876a0d3137a221f2bc85354971678f015e5e61756064bd7cdc10d5"} Apr 16 18:16:12.247684 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:12.247619 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:12.247684 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:12.247689 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:12.247916 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:12.247780 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 18:16:12.247916 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:12.247852 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert podName:1f3ce71d-67a7-431e-9a65-37dd55851b2c nodeName:}" failed. No retries permitted until 2026-04-16 18:16:14.247833413 +0000 UTC m=+38.641214647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert") pod "ingress-canary-fqnj7" (UID: "1f3ce71d-67a7-431e-9a65-37dd55851b2c") : secret "canary-serving-cert" not found Apr 16 18:16:12.247916 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:12.247779 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 18:16:12.248021 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:12.247925 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls podName:e7a02fec-362b-4be1-bf77-0fbf2a0a986e nodeName:}" failed. No retries permitted until 2026-04-16 18:16:14.247911612 +0000 UTC m=+38.641292847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls") pod "dns-default-7mtp7" (UID: "e7a02fec-362b-4be1-bf77-0fbf2a0a986e") : secret "dns-default-metrics-tls" not found Apr 16 18:16:12.417231 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:12.417199 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59lzl" event={"ID":"9db69d07-9153-4912-af0e-5a73aa0f3f33","Type":"ContainerStarted","Data":"f73ec6c8b18c9dcf822fdcde50652f6a51c747338b19ad035970eea8868e1e84"} Apr 16 18:16:12.441602 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:12.441554 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-59lzl" podStartSLOduration=5.271425941 podStartE2EDuration="36.441541221s" podCreationTimestamp="2026-04-16 18:15:36 +0000 UTC" firstStartedPulling="2026-04-16 18:15:38.829608743 +0000 UTC m=+3.222989981" lastFinishedPulling="2026-04-16 18:16:09.999724021 +0000 UTC m=+34.393105261" observedRunningTime="2026-04-16 18:16:12.440720821 +0000 UTC m=+36.834102078" watchObservedRunningTime="2026-04-16 18:16:12.441541221 +0000 UTC m=+36.834922478" Apr 16 18:16:14.263558 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:14.263502 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:14.263558 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:14.263556 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:14.264013 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:14.263662 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 18:16:14.264013 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:14.263664 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 18:16:14.264013 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:14.263713 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls podName:e7a02fec-362b-4be1-bf77-0fbf2a0a986e nodeName:}" failed. No retries permitted until 2026-04-16 18:16:18.26370046 +0000 UTC m=+42.657081699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls") pod "dns-default-7mtp7" (UID: "e7a02fec-362b-4be1-bf77-0fbf2a0a986e") : secret "dns-default-metrics-tls" not found Apr 16 18:16:14.264013 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:14.263727 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert podName:1f3ce71d-67a7-431e-9a65-37dd55851b2c nodeName:}" failed. No retries permitted until 2026-04-16 18:16:18.263719923 +0000 UTC m=+42.657101162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert") pod "ingress-canary-fqnj7" (UID: "1f3ce71d-67a7-431e-9a65-37dd55851b2c") : secret "canary-serving-cert" not found Apr 16 18:16:18.291182 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:18.291125 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:18.291724 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:18.291225 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:18.291724 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:18.291277 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 18:16:18.291724 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:18.291302 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 18:16:18.291724 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:18.291341 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls podName:e7a02fec-362b-4be1-bf77-0fbf2a0a986e nodeName:}" failed. No retries permitted until 2026-04-16 18:16:26.291326032 +0000 UTC m=+50.684707267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls") pod "dns-default-7mtp7" (UID: "e7a02fec-362b-4be1-bf77-0fbf2a0a986e") : secret "dns-default-metrics-tls" not found Apr 16 18:16:18.291724 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:18.291355 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert podName:1f3ce71d-67a7-431e-9a65-37dd55851b2c nodeName:}" failed. No retries permitted until 2026-04-16 18:16:26.291349074 +0000 UTC m=+50.684730310 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert") pod "ingress-canary-fqnj7" (UID: "1f3ce71d-67a7-431e-9a65-37dd55851b2c") : secret "canary-serving-cert" not found Apr 16 18:16:26.345606 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:26.345561 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:26.345606 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:26.345607 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:26.346176 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:26.345740 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 18:16:26.346176 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:26.345745 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 18:16:26.346176 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:26.345802 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls podName:e7a02fec-362b-4be1-bf77-0fbf2a0a986e nodeName:}" failed. No retries permitted until 2026-04-16 18:16:42.34578734 +0000 UTC m=+66.739168574 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls") pod "dns-default-7mtp7" (UID: "e7a02fec-362b-4be1-bf77-0fbf2a0a986e") : secret "dns-default-metrics-tls" not found Apr 16 18:16:26.346176 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:26.345816 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert podName:1f3ce71d-67a7-431e-9a65-37dd55851b2c nodeName:}" failed. No retries permitted until 2026-04-16 18:16:42.345809755 +0000 UTC m=+66.739190990 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert") pod "ingress-canary-fqnj7" (UID: "1f3ce71d-67a7-431e-9a65-37dd55851b2c") : secret "canary-serving-cert" not found Apr 16 18:16:35.408748 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:35.408715 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgfh9" Apr 16 18:16:41.860503 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:41.860469 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7pq5\" (UniqueName: \"kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5\") pod \"network-check-target-zf6t5\" (UID: \"9792d3fc-f6a7-4d56-a906-b650dea53fb2\") " pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:41.860503 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:41.860508 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:16:41.862874 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:41.862853 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 16 18:16:41.862945 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:41.862853 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 16 18:16:41.870675 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:41.870653 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 16 18:16:41.870801 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:41.870712 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs podName:5609e075-d6f5-4642-a711-20f608288876 nodeName:}" failed. No retries permitted until 2026-04-16 18:17:45.870697497 +0000 UTC m=+130.264078732 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs") pod "network-metrics-daemon-lp8x6" (UID: "5609e075-d6f5-4642-a711-20f608288876") : secret "metrics-daemon-secret" not found Apr 16 18:16:41.872591 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:41.872577 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 16 18:16:41.883852 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:41.883829 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7pq5\" (UniqueName: \"kubernetes.io/projected/9792d3fc-f6a7-4d56-a906-b650dea53fb2-kube-api-access-m7pq5\") pod \"network-check-target-zf6t5\" (UID: \"9792d3fc-f6a7-4d56-a906-b650dea53fb2\") " pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:42.127332 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:42.127241 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xl8bl\"" Apr 16 18:16:42.135168 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:42.135141 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:42.267378 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:42.267321 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zf6t5"] Apr 16 18:16:42.271775 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:16:42.271749 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9792d3fc_f6a7_4d56_a906_b650dea53fb2.slice/crio-800afbf0ba692cfd18c85c4df4ec6f55c0547bc859a871efde2a38df21ebeb9c WatchSource:0}: Error finding container 800afbf0ba692cfd18c85c4df4ec6f55c0547bc859a871efde2a38df21ebeb9c: Status 404 returned error can't find the container with id 800afbf0ba692cfd18c85c4df4ec6f55c0547bc859a871efde2a38df21ebeb9c Apr 16 18:16:42.364028 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:42.363981 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:16:42.364028 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:42.364030 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:16:42.364271 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:42.364146 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 18:16:42.364271 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:42.364149 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 18:16:42.364271 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:42.364220 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert podName:1f3ce71d-67a7-431e-9a65-37dd55851b2c nodeName:}" failed. No retries permitted until 2026-04-16 18:17:14.36420471 +0000 UTC m=+98.757585946 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert") pod "ingress-canary-fqnj7" (UID: "1f3ce71d-67a7-431e-9a65-37dd55851b2c") : secret "canary-serving-cert" not found Apr 16 18:16:42.364271 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:16:42.364236 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls podName:e7a02fec-362b-4be1-bf77-0fbf2a0a986e nodeName:}" failed. No retries permitted until 2026-04-16 18:17:14.364228951 +0000 UTC m=+98.757610186 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls") pod "dns-default-7mtp7" (UID: "e7a02fec-362b-4be1-bf77-0fbf2a0a986e") : secret "dns-default-metrics-tls" not found Apr 16 18:16:42.474907 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:42.474822 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zf6t5" event={"ID":"9792d3fc-f6a7-4d56-a906-b650dea53fb2","Type":"ContainerStarted","Data":"800afbf0ba692cfd18c85c4df4ec6f55c0547bc859a871efde2a38df21ebeb9c"} Apr 16 18:16:45.988653 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:45.988598 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f"] Apr 16 18:16:45.991823 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:45.991806 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:45.997642 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:45.997594 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Apr 16 18:16:45.997642 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:45.997618 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Apr 16 18:16:45.997977 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:45.997620 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Apr 16 18:16:45.997977 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:45.997676 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Apr 16 18:16:45.997977 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:45.997937 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Apr 16 18:16:45.998239 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:45.998132 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Apr 16 18:16:45.998297 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:45.998233 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Apr 16 18:16:46.003608 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.003585 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f"] Apr 16 18:16:46.093012 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.092969 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-ca\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.093012 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.093008 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-hub\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.093228 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.093046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.093228 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.093074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.093228 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.093121 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9ttp\" (UniqueName: \"kubernetes.io/projected/a916945b-958f-4d5d-b8c3-ba3362db924b-kube-api-access-s9ttp\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.093228 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.093157 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/a916945b-958f-4d5d-b8c3-ba3362db924b-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.194346 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.194296 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.194346 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.194354 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s9ttp\" (UniqueName: \"kubernetes.io/projected/a916945b-958f-4d5d-b8c3-ba3362db924b-kube-api-access-s9ttp\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.194598 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.194389 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/a916945b-958f-4d5d-b8c3-ba3362db924b-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.194598 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.194414 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-ca\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.194598 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.194432 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-hub\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.194598 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.194535 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.195384 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.195351 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/a916945b-958f-4d5d-b8c3-ba3362db924b-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.197092 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.197071 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-hub\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.197212 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.197133 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-ca\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.197307 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.197289 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.201944 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.201923 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a916945b-958f-4d5d-b8c3-ba3362db924b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.203266 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.203237 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9ttp\" (UniqueName: \"kubernetes.io/projected/a916945b-958f-4d5d-b8c3-ba3362db924b-kube-api-access-s9ttp\") pod \"cluster-proxy-proxy-agent-6847cbfdb5-ww89f\" (UID: \"a916945b-958f-4d5d-b8c3-ba3362db924b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.314541 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.314503 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:16:46.452090 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.452054 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f"] Apr 16 18:16:46.455795 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:16:46.455755 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda916945b_958f_4d5d_b8c3_ba3362db924b.slice/crio-3446e43b5d032835d8760beeab5b05d3286187603c5fc0a2e1081a6fa8aae7f6 WatchSource:0}: Error finding container 3446e43b5d032835d8760beeab5b05d3286187603c5fc0a2e1081a6fa8aae7f6: Status 404 returned error can't find the container with id 3446e43b5d032835d8760beeab5b05d3286187603c5fc0a2e1081a6fa8aae7f6 Apr 16 18:16:46.484832 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.484790 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zf6t5" event={"ID":"9792d3fc-f6a7-4d56-a906-b650dea53fb2","Type":"ContainerStarted","Data":"f6ded46b3019b25e068374d0f4d27c299abbe15bc9162d638d2ecb3173643ffa"} Apr 16 18:16:46.485020 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.485004 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:16:46.485867 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.485845 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" event={"ID":"a916945b-958f-4d5d-b8c3-ba3362db924b","Type":"ContainerStarted","Data":"3446e43b5d032835d8760beeab5b05d3286187603c5fc0a2e1081a6fa8aae7f6"} Apr 16 18:16:46.505229 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:46.505185 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zf6t5" podStartSLOduration=67.40652057 podStartE2EDuration="1m10.505172652s" podCreationTimestamp="2026-04-16 18:15:36 +0000 UTC" firstStartedPulling="2026-04-16 18:16:42.27354029 +0000 UTC m=+66.666921525" lastFinishedPulling="2026-04-16 18:16:45.372192358 +0000 UTC m=+69.765573607" observedRunningTime="2026-04-16 18:16:46.504471113 +0000 UTC m=+70.897852364" watchObservedRunningTime="2026-04-16 18:16:46.505172652 +0000 UTC m=+70.898553903" Apr 16 18:16:49.494577 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:49.494541 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" event={"ID":"a916945b-958f-4d5d-b8c3-ba3362db924b","Type":"ContainerStarted","Data":"ea51e2ea6b76693486a2946d3ca4ee9bb0ca68ec68aadd82ead61b3937a0ab4f"} Apr 16 18:16:51.500923 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:51.500885 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" event={"ID":"a916945b-958f-4d5d-b8c3-ba3362db924b","Type":"ContainerStarted","Data":"e735af79f467c23656db0a8087ad1622e57ca4caa2ae9021ab0022f599ec79e6"} Apr 16 18:16:51.500923 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:51.500929 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" event={"ID":"a916945b-958f-4d5d-b8c3-ba3362db924b","Type":"ContainerStarted","Data":"1c180571f899dd63f09383a5a368b1d32fec218466aaa1f9bb4d46b9ac5f161d"} Apr 16 18:16:51.520745 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:16:51.520695 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" podStartSLOduration=2.251735124 podStartE2EDuration="6.520681581s" podCreationTimestamp="2026-04-16 18:16:45 +0000 UTC" firstStartedPulling="2026-04-16 18:16:46.457567233 +0000 UTC m=+70.850948473" lastFinishedPulling="2026-04-16 18:16:50.726513693 +0000 UTC m=+75.119894930" observedRunningTime="2026-04-16 18:16:51.519967418 +0000 UTC m=+75.913348675" watchObservedRunningTime="2026-04-16 18:16:51.520681581 +0000 UTC m=+75.914062883" Apr 16 18:17:14.392003 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:14.391938 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:17:14.392003 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:14.392002 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:17:14.392483 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:14.392086 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 18:17:14.392483 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:14.392090 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 18:17:14.392483 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:14.392150 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls podName:e7a02fec-362b-4be1-bf77-0fbf2a0a986e nodeName:}" failed. No retries permitted until 2026-04-16 18:18:18.392135302 +0000 UTC m=+162.785516537 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls") pod "dns-default-7mtp7" (UID: "e7a02fec-362b-4be1-bf77-0fbf2a0a986e") : secret "dns-default-metrics-tls" not found Apr 16 18:17:14.392483 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:14.392163 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert podName:1f3ce71d-67a7-431e-9a65-37dd55851b2c nodeName:}" failed. No retries permitted until 2026-04-16 18:18:18.392157824 +0000 UTC m=+162.785539059 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert") pod "ingress-canary-fqnj7" (UID: "1f3ce71d-67a7-431e-9a65-37dd55851b2c") : secret "canary-serving-cert" not found Apr 16 18:17:17.490544 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:17.490512 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zf6t5" Apr 16 18:17:45.904120 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:45.904069 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:17:45.904604 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:45.904244 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 16 18:17:45.904604 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:45.904331 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs podName:5609e075-d6f5-4642-a711-20f608288876 nodeName:}" failed. No retries permitted until 2026-04-16 18:19:47.904313751 +0000 UTC m=+252.297694986 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs") pod "network-metrics-daemon-lp8x6" (UID: "5609e075-d6f5-4642-a711-20f608288876") : secret "metrics-daemon-secret" not found Apr 16 18:17:50.084287 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.084252 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-d87b8d5fc-l9q5d"] Apr 16 18:17:50.086069 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.086054 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.088509 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.088475 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Apr 16 18:17:50.088657 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.088571 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hz7nr\"" Apr 16 18:17:50.088999 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.088984 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Apr 16 18:17:50.089264 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.089242 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Apr 16 18:17:50.089510 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.089496 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Apr 16 18:17:50.099276 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.099252 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Apr 16 18:17:50.099512 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.099494 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-d87b8d5fc-l9q5d"] Apr 16 18:17:50.234994 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.234946 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/14efee5f-03ef-4047-86f8-ede90b8b5238-serving-cert\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.235182 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.235013 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/14efee5f-03ef-4047-86f8-ede90b8b5238-config\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.235182 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.235041 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7w77\" (UniqueName: \"kubernetes.io/projected/14efee5f-03ef-4047-86f8-ede90b8b5238-kube-api-access-p7w77\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.235182 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.235093 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/14efee5f-03ef-4047-86f8-ede90b8b5238-trusted-ca\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.336357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.336255 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p7w77\" (UniqueName: \"kubernetes.io/projected/14efee5f-03ef-4047-86f8-ede90b8b5238-kube-api-access-p7w77\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.336357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.336358 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/14efee5f-03ef-4047-86f8-ede90b8b5238-trusted-ca\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.336537 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.336414 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/14efee5f-03ef-4047-86f8-ede90b8b5238-serving-cert\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.336537 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.336449 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/14efee5f-03ef-4047-86f8-ede90b8b5238-config\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.337138 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.337114 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/14efee5f-03ef-4047-86f8-ede90b8b5238-config\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.337225 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.337206 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/14efee5f-03ef-4047-86f8-ede90b8b5238-trusted-ca\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.338966 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.338950 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/14efee5f-03ef-4047-86f8-ede90b8b5238-serving-cert\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.344195 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.344167 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7w77\" (UniqueName: \"kubernetes.io/projected/14efee5f-03ef-4047-86f8-ede90b8b5238-kube-api-access-p7w77\") pod \"console-operator-d87b8d5fc-l9q5d\" (UID: \"14efee5f-03ef-4047-86f8-ede90b8b5238\") " pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.395188 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.395147 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:17:50.511606 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.511576 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-d87b8d5fc-l9q5d"] Apr 16 18:17:50.514642 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:17:50.514594 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14efee5f_03ef_4047_86f8_ede90b8b5238.slice/crio-32fa12780f54e2928c6a18396d8194e24ee8e19ed7967b82c742a609c0b08992 WatchSource:0}: Error finding container 32fa12780f54e2928c6a18396d8194e24ee8e19ed7967b82c742a609c0b08992: Status 404 returned error can't find the container with id 32fa12780f54e2928c6a18396d8194e24ee8e19ed7967b82c742a609c0b08992 Apr 16 18:17:50.611589 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:50.611503 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" event={"ID":"14efee5f-03ef-4047-86f8-ede90b8b5238","Type":"ContainerStarted","Data":"32fa12780f54e2928c6a18396d8194e24ee8e19ed7967b82c742a609c0b08992"} Apr 16 18:17:52.379745 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:52.379716 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kdfqc_59928e2c-472c-4289-b77d-213f2811bb82/dns-node-resolver/0.log" Apr 16 18:17:52.616643 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:52.616601 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/0.log" Apr 16 18:17:52.616809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:52.616665 2577 generic.go:358] "Generic (PLEG): container finished" podID="14efee5f-03ef-4047-86f8-ede90b8b5238" containerID="37ee07d035b0c77d41ea5a99de49e363849b03bec65097ec8c1a427f39be0605" exitCode=255 Apr 16 18:17:52.616809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:52.616744 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" event={"ID":"14efee5f-03ef-4047-86f8-ede90b8b5238","Type":"ContainerDied","Data":"37ee07d035b0c77d41ea5a99de49e363849b03bec65097ec8c1a427f39be0605"} Apr 16 18:17:52.616977 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:52.616964 2577 scope.go:117] "RemoveContainer" containerID="37ee07d035b0c77d41ea5a99de49e363849b03bec65097ec8c1a427f39be0605" Apr 16 18:17:53.177022 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:53.176993 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wh4nz_109afb78-2acc-4972-9604-32fb10a7ae6b/node-ca/0.log" Apr 16 18:17:53.620640 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:53.620598 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/1.log" Apr 16 18:17:53.621069 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:53.621017 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/0.log" Apr 16 18:17:53.621069 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:53.621057 2577 generic.go:358] "Generic (PLEG): container finished" podID="14efee5f-03ef-4047-86f8-ede90b8b5238" containerID="e9b9b27076b37638f1912181560694a6b46aa323153ce31cab96a22f85352ed0" exitCode=255 Apr 16 18:17:53.621178 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:53.621123 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" event={"ID":"14efee5f-03ef-4047-86f8-ede90b8b5238","Type":"ContainerDied","Data":"e9b9b27076b37638f1912181560694a6b46aa323153ce31cab96a22f85352ed0"} Apr 16 18:17:53.621178 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:53.621169 2577 scope.go:117] "RemoveContainer" containerID="37ee07d035b0c77d41ea5a99de49e363849b03bec65097ec8c1a427f39be0605" Apr 16 18:17:53.621364 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:53.621346 2577 scope.go:117] "RemoveContainer" containerID="e9b9b27076b37638f1912181560694a6b46aa323153ce31cab96a22f85352ed0" Apr 16 18:17:53.621553 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:53.621527 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-d87b8d5fc-l9q5d_openshift-console-operator(14efee5f-03ef-4047-86f8-ede90b8b5238)\"" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" podUID="14efee5f-03ef-4047-86f8-ede90b8b5238" Apr 16 18:17:54.624729 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:54.624699 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/1.log" Apr 16 18:17:54.625102 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:54.625047 2577 scope.go:117] "RemoveContainer" containerID="e9b9b27076b37638f1912181560694a6b46aa323153ce31cab96a22f85352ed0" Apr 16 18:17:54.625229 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:54.625210 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-d87b8d5fc-l9q5d_openshift-console-operator(14efee5f-03ef-4047-86f8-ede90b8b5238)\"" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" podUID="14efee5f-03ef-4047-86f8-ede90b8b5238" Apr 16 18:17:56.098661 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.098616 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r"] Apr 16 18:17:56.101351 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.101333 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" Apr 16 18:17:56.103652 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.103599 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Apr 16 18:17:56.103652 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.103607 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-fbsm2\"" Apr 16 18:17:56.103827 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.103607 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Apr 16 18:17:56.104238 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.104217 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Apr 16 18:17:56.104449 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.104432 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Apr 16 18:17:56.109529 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.109510 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r"] Apr 16 18:17:56.181072 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.181024 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df11f2db-bfe9-48ff-a2f7-cfc24826b010-config\") pod \"service-ca-operator-69965bb79d-6sb7r\" (UID: \"df11f2db-bfe9-48ff-a2f7-cfc24826b010\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" Apr 16 18:17:56.181072 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.181072 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd75q\" (UniqueName: \"kubernetes.io/projected/df11f2db-bfe9-48ff-a2f7-cfc24826b010-kube-api-access-zd75q\") pod \"service-ca-operator-69965bb79d-6sb7r\" (UID: \"df11f2db-bfe9-48ff-a2f7-cfc24826b010\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" Apr 16 18:17:56.181312 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.181148 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df11f2db-bfe9-48ff-a2f7-cfc24826b010-serving-cert\") pod \"service-ca-operator-69965bb79d-6sb7r\" (UID: \"df11f2db-bfe9-48ff-a2f7-cfc24826b010\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" Apr 16 18:17:56.281528 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.281491 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df11f2db-bfe9-48ff-a2f7-cfc24826b010-config\") pod \"service-ca-operator-69965bb79d-6sb7r\" (UID: \"df11f2db-bfe9-48ff-a2f7-cfc24826b010\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" Apr 16 18:17:56.281528 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.281530 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zd75q\" (UniqueName: \"kubernetes.io/projected/df11f2db-bfe9-48ff-a2f7-cfc24826b010-kube-api-access-zd75q\") pod \"service-ca-operator-69965bb79d-6sb7r\" (UID: \"df11f2db-bfe9-48ff-a2f7-cfc24826b010\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" Apr 16 18:17:56.281729 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.281569 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df11f2db-bfe9-48ff-a2f7-cfc24826b010-serving-cert\") pod \"service-ca-operator-69965bb79d-6sb7r\" (UID: \"df11f2db-bfe9-48ff-a2f7-cfc24826b010\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" Apr 16 18:17:56.282043 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.282021 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df11f2db-bfe9-48ff-a2f7-cfc24826b010-config\") pod \"service-ca-operator-69965bb79d-6sb7r\" (UID: \"df11f2db-bfe9-48ff-a2f7-cfc24826b010\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" Apr 16 18:17:56.283975 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.283947 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df11f2db-bfe9-48ff-a2f7-cfc24826b010-serving-cert\") pod \"service-ca-operator-69965bb79d-6sb7r\" (UID: \"df11f2db-bfe9-48ff-a2f7-cfc24826b010\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" Apr 16 18:17:56.292464 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.292440 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd75q\" (UniqueName: \"kubernetes.io/projected/df11f2db-bfe9-48ff-a2f7-cfc24826b010-kube-api-access-zd75q\") pod \"service-ca-operator-69965bb79d-6sb7r\" (UID: \"df11f2db-bfe9-48ff-a2f7-cfc24826b010\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" Apr 16 18:17:56.410845 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.410758 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" Apr 16 18:17:56.532531 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.532466 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r"] Apr 16 18:17:56.536803 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:17:56.536774 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf11f2db_bfe9_48ff_a2f7_cfc24826b010.slice/crio-a9d2dae4fdecce3ca7a7419394e84417c354739958d3a3eb0a08e0c67a9c595f WatchSource:0}: Error finding container a9d2dae4fdecce3ca7a7419394e84417c354739958d3a3eb0a08e0c67a9c595f: Status 404 returned error can't find the container with id a9d2dae4fdecce3ca7a7419394e84417c354739958d3a3eb0a08e0c67a9c595f Apr 16 18:17:56.629482 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:56.629444 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" event={"ID":"df11f2db-bfe9-48ff-a2f7-cfc24826b010","Type":"ContainerStarted","Data":"a9d2dae4fdecce3ca7a7419394e84417c354739958d3a3eb0a08e0c67a9c595f"} Apr 16 18:17:57.906776 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.906735 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79699d8d5-9zpdb"] Apr 16 18:17:57.910261 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.910239 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:57.912455 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.912433 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Apr 16 18:17:57.912455 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.912444 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Apr 16 18:17:57.912772 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.912756 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-846cr\"" Apr 16 18:17:57.912833 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.912770 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Apr 16 18:17:57.926549 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.926522 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Apr 16 18:17:57.929896 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.929862 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79699d8d5-9zpdb"] Apr 16 18:17:57.993491 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.993445 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:57.993704 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.993517 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-image-registry-private-configuration\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:57.993704 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.993545 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtd5v\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-kube-api-access-gtd5v\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:57.993704 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.993602 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-trusted-ca\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:57.993704 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.993653 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-ca-trust-extracted\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:57.993909 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.993712 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-installation-pull-secrets\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:57.993909 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.993750 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-certificates\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:57.993909 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:57.993773 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-bound-sa-token\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.094211 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.094168 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.094211 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.094228 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-image-registry-private-configuration\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.094471 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.094256 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gtd5v\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-kube-api-access-gtd5v\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.094471 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.094289 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-trusted-ca\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.094471 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.094317 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-ca-trust-extracted\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.094471 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:58.094332 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 18:17:58.094471 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:58.094349 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79699d8d5-9zpdb: secret "image-registry-tls" not found Apr 16 18:17:58.094471 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.094370 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-installation-pull-secrets\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.094471 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.094397 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-certificates\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.094471 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:58.094423 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls podName:2139e7d8-f98a-49e0-a5a4-1d715b1a9a94 nodeName:}" failed. No retries permitted until 2026-04-16 18:17:58.594401011 +0000 UTC m=+142.987782260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls") pod "image-registry-79699d8d5-9zpdb" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94") : secret "image-registry-tls" not found Apr 16 18:17:58.094907 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.094696 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-bound-sa-token\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.095061 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.095041 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-certificates\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.095148 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.095071 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-ca-trust-extracted\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.095533 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.095511 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-trusted-ca\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.097140 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.097101 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-image-registry-private-configuration\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.097534 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.097511 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-installation-pull-secrets\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.119446 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.119413 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtd5v\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-kube-api-access-gtd5v\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.119832 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.119810 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-bound-sa-token\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.598834 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.598782 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:58.599018 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:58.598919 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 18:17:58.599018 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:58.598940 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79699d8d5-9zpdb: secret "image-registry-tls" not found Apr 16 18:17:58.599018 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:58.598993 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls podName:2139e7d8-f98a-49e0-a5a4-1d715b1a9a94 nodeName:}" failed. No retries permitted until 2026-04-16 18:17:59.598975072 +0000 UTC m=+143.992356328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls") pod "image-registry-79699d8d5-9zpdb" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94") : secret "image-registry-tls" not found Apr 16 18:17:58.636077 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.636003 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" event={"ID":"df11f2db-bfe9-48ff-a2f7-cfc24826b010","Type":"ContainerStarted","Data":"1500b8d59934c62ee1846ba19cfdf0faec68aa6de9607a89460d8e0d577dc4cc"} Apr 16 18:17:58.659008 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:58.658957 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" podStartSLOduration=0.817971814 podStartE2EDuration="2.658940138s" podCreationTimestamp="2026-04-16 18:17:56 +0000 UTC" firstStartedPulling="2026-04-16 18:17:56.538761862 +0000 UTC m=+140.932143110" lastFinishedPulling="2026-04-16 18:17:58.379730185 +0000 UTC m=+142.773111434" observedRunningTime="2026-04-16 18:17:58.658441473 +0000 UTC m=+143.051822731" watchObservedRunningTime="2026-04-16 18:17:58.658940138 +0000 UTC m=+143.052321394" Apr 16 18:17:59.607356 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:17:59.607316 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:17:59.607781 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:59.607496 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 18:17:59.607781 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:59.607524 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79699d8d5-9zpdb: secret "image-registry-tls" not found Apr 16 18:17:59.607781 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:17:59.607598 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls podName:2139e7d8-f98a-49e0-a5a4-1d715b1a9a94 nodeName:}" failed. No retries permitted until 2026-04-16 18:18:01.607575909 +0000 UTC m=+146.000957149 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls") pod "image-registry-79699d8d5-9zpdb" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94") : secret "image-registry-tls" not found Apr 16 18:18:00.395426 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:00.395388 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:18:00.395426 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:00.395435 2577 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:18:00.395818 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:00.395807 2577 scope.go:117] "RemoveContainer" containerID="e9b9b27076b37638f1912181560694a6b46aa323153ce31cab96a22f85352ed0" Apr 16 18:18:00.395997 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:00.395982 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-d87b8d5fc-l9q5d_openshift-console-operator(14efee5f-03ef-4047-86f8-ede90b8b5238)\"" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" podUID="14efee5f-03ef-4047-86f8-ede90b8b5238" Apr 16 18:18:01.623399 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:01.623353 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:18:01.623811 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:01.623466 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 18:18:01.623811 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:01.623478 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79699d8d5-9zpdb: secret "image-registry-tls" not found Apr 16 18:18:01.623811 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:01.623539 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls podName:2139e7d8-f98a-49e0-a5a4-1d715b1a9a94 nodeName:}" failed. No retries permitted until 2026-04-16 18:18:05.623524694 +0000 UTC m=+150.016905929 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls") pod "image-registry-79699d8d5-9zpdb" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94") : secret "image-registry-tls" not found Apr 16 18:18:05.656258 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:05.656216 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:18:05.656670 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:05.656365 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 18:18:05.656670 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:05.656383 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79699d8d5-9zpdb: secret "image-registry-tls" not found Apr 16 18:18:05.656670 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:05.656441 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls podName:2139e7d8-f98a-49e0-a5a4-1d715b1a9a94 nodeName:}" failed. No retries permitted until 2026-04-16 18:18:13.656425338 +0000 UTC m=+158.049806572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls") pod "image-registry-79699d8d5-9zpdb" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94") : secret "image-registry-tls" not found Apr 16 18:18:13.210024 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.209924 2577 scope.go:117] "RemoveContainer" containerID="e9b9b27076b37638f1912181560694a6b46aa323153ce31cab96a22f85352ed0" Apr 16 18:18:13.548722 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:13.548674 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-fqnj7" podUID="1f3ce71d-67a7-431e-9a65-37dd55851b2c" Apr 16 18:18:13.555860 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:13.555828 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7mtp7" podUID="e7a02fec-362b-4be1-bf77-0fbf2a0a986e" Apr 16 18:18:13.673490 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.673463 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/2.log" Apr 16 18:18:13.673862 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.673846 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/1.log" Apr 16 18:18:13.673910 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.673883 2577 generic.go:358] "Generic (PLEG): container finished" podID="14efee5f-03ef-4047-86f8-ede90b8b5238" containerID="f360c7000c5663ce62b5196f7691bbecbd2e753923e00057d91033476f3448cd" exitCode=255 Apr 16 18:18:13.673982 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.673962 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" event={"ID":"14efee5f-03ef-4047-86f8-ede90b8b5238","Type":"ContainerDied","Data":"f360c7000c5663ce62b5196f7691bbecbd2e753923e00057d91033476f3448cd"} Apr 16 18:18:13.674018 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.674006 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7mtp7" Apr 16 18:18:13.674089 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.674006 2577 scope.go:117] "RemoveContainer" containerID="e9b9b27076b37638f1912181560694a6b46aa323153ce31cab96a22f85352ed0" Apr 16 18:18:13.674413 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.674392 2577 scope.go:117] "RemoveContainer" containerID="f360c7000c5663ce62b5196f7691bbecbd2e753923e00057d91033476f3448cd" Apr 16 18:18:13.674647 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:13.674606 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-d87b8d5fc-l9q5d_openshift-console-operator(14efee5f-03ef-4047-86f8-ede90b8b5238)\"" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" podUID="14efee5f-03ef-4047-86f8-ede90b8b5238" Apr 16 18:18:13.720216 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.720174 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:18:13.722722 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.722702 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls\") pod \"image-registry-79699d8d5-9zpdb\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:18:13.821521 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.821436 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:18:13.944888 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:13.944857 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79699d8d5-9zpdb"] Apr 16 18:18:13.947678 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:18:13.947649 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2139e7d8_f98a_49e0_a5a4_1d715b1a9a94.slice/crio-733434ce9c863b5c6d158eaf1348d9c2db6981ebb396ad002d52e4d0528f7258 WatchSource:0}: Error finding container 733434ce9c863b5c6d158eaf1348d9c2db6981ebb396ad002d52e4d0528f7258: Status 404 returned error can't find the container with id 733434ce9c863b5c6d158eaf1348d9c2db6981ebb396ad002d52e4d0528f7258 Apr 16 18:18:14.220276 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:14.220176 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-lp8x6" podUID="5609e075-d6f5-4642-a711-20f608288876" Apr 16 18:18:14.677371 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:14.677331 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" event={"ID":"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94","Type":"ContainerStarted","Data":"8f6960f6e1ef603acc782bd4e49541c06efe96df274337834ecff2820b8bfa42"} Apr 16 18:18:14.677371 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:14.677375 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" event={"ID":"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94","Type":"ContainerStarted","Data":"733434ce9c863b5c6d158eaf1348d9c2db6981ebb396ad002d52e4d0528f7258"} Apr 16 18:18:14.677621 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:14.677572 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:18:14.678724 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:14.678707 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/2.log" Apr 16 18:18:14.699721 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:14.699673 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" podStartSLOduration=17.699659156 podStartE2EDuration="17.699659156s" podCreationTimestamp="2026-04-16 18:17:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 18:18:14.698495522 +0000 UTC m=+159.091876778" watchObservedRunningTime="2026-04-16 18:18:14.699659156 +0000 UTC m=+159.093040412" Apr 16 18:18:16.315917 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:16.315852 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" podUID="a916945b-958f-4d5d-b8c3-ba3362db924b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Apr 16 18:18:18.460686 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:18.460618 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:18:18.460686 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:18.460687 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:18:18.463368 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:18.463337 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7a02fec-362b-4be1-bf77-0fbf2a0a986e-metrics-tls\") pod \"dns-default-7mtp7\" (UID: \"e7a02fec-362b-4be1-bf77-0fbf2a0a986e\") " pod="openshift-dns/dns-default-7mtp7" Apr 16 18:18:18.463468 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:18.463379 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f3ce71d-67a7-431e-9a65-37dd55851b2c-cert\") pod \"ingress-canary-fqnj7\" (UID: \"1f3ce71d-67a7-431e-9a65-37dd55851b2c\") " pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:18:18.477848 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:18.477821 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fxg25\"" Apr 16 18:18:18.485710 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:18.485681 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7mtp7" Apr 16 18:18:18.608285 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:18.608250 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7mtp7"] Apr 16 18:18:18.611440 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:18:18.611412 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a02fec_362b_4be1_bf77_0fbf2a0a986e.slice/crio-ddec1f55066601b347ecaee985bcc915c92b72a37fb26cf3b9b6f43bf23432f0 WatchSource:0}: Error finding container ddec1f55066601b347ecaee985bcc915c92b72a37fb26cf3b9b6f43bf23432f0: Status 404 returned error can't find the container with id ddec1f55066601b347ecaee985bcc915c92b72a37fb26cf3b9b6f43bf23432f0 Apr 16 18:18:18.689094 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:18.689060 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7mtp7" event={"ID":"e7a02fec-362b-4be1-bf77-0fbf2a0a986e","Type":"ContainerStarted","Data":"ddec1f55066601b347ecaee985bcc915c92b72a37fb26cf3b9b6f43bf23432f0"} Apr 16 18:18:20.395996 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:20.395936 2577 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:18:20.395996 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:20.395999 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:18:20.396431 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:20.396391 2577 scope.go:117] "RemoveContainer" containerID="f360c7000c5663ce62b5196f7691bbecbd2e753923e00057d91033476f3448cd" Apr 16 18:18:20.396617 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:20.396598 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-d87b8d5fc-l9q5d_openshift-console-operator(14efee5f-03ef-4047-86f8-ede90b8b5238)\"" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" podUID="14efee5f-03ef-4047-86f8-ede90b8b5238" Apr 16 18:18:20.695472 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:20.695376 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7mtp7" event={"ID":"e7a02fec-362b-4be1-bf77-0fbf2a0a986e","Type":"ContainerStarted","Data":"c30be29a1c0aa0f6f92356e01d92c07aff95ee7197ac094afe50a7a6ce7ad07a"} Apr 16 18:18:20.695472 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:20.695415 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7mtp7" event={"ID":"e7a02fec-362b-4be1-bf77-0fbf2a0a986e","Type":"ContainerStarted","Data":"da3d63bbe21a9b847ddfaa99cac95dc991fcd891e18ac475114c0998932bb966"} Apr 16 18:18:20.695730 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:20.695588 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7mtp7" Apr 16 18:18:20.714369 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:20.714317 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-7mtp7" podStartSLOduration=129.49130773 podStartE2EDuration="2m10.714299568s" podCreationTimestamp="2026-04-16 18:16:10 +0000 UTC" firstStartedPulling="2026-04-16 18:18:18.61320097 +0000 UTC m=+163.006582205" lastFinishedPulling="2026-04-16 18:18:19.836192808 +0000 UTC m=+164.229574043" observedRunningTime="2026-04-16 18:18:20.714075434 +0000 UTC m=+165.107456692" watchObservedRunningTime="2026-04-16 18:18:20.714299568 +0000 UTC m=+165.107680825" Apr 16 18:18:21.524427 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.524392 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4rkbt"] Apr 16 18:18:21.527298 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.527273 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.529642 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.529605 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Apr 16 18:18:21.530265 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.530239 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Apr 16 18:18:21.530517 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.530500 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Apr 16 18:18:21.530583 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.530522 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Apr 16 18:18:21.530583 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.530550 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-czcrj\"" Apr 16 18:18:21.541121 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.541086 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4rkbt"] Apr 16 18:18:21.601089 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.601054 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79699d8d5-9zpdb"] Apr 16 18:18:21.684052 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.684019 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/203100f0-5c6d-4372-a64b-f2823d3d7d55-crio-socket\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.684222 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.684056 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/203100f0-5c6d-4372-a64b-f2823d3d7d55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.684222 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.684086 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgnp5\" (UniqueName: \"kubernetes.io/projected/203100f0-5c6d-4372-a64b-f2823d3d7d55-kube-api-access-hgnp5\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.684222 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.684122 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/203100f0-5c6d-4372-a64b-f2823d3d7d55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.684222 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.684168 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/203100f0-5c6d-4372-a64b-f2823d3d7d55-data-volume\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.785155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.785062 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hgnp5\" (UniqueName: \"kubernetes.io/projected/203100f0-5c6d-4372-a64b-f2823d3d7d55-kube-api-access-hgnp5\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.785155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.785124 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/203100f0-5c6d-4372-a64b-f2823d3d7d55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.785155 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.785154 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/203100f0-5c6d-4372-a64b-f2823d3d7d55-data-volume\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.785416 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.785228 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/203100f0-5c6d-4372-a64b-f2823d3d7d55-crio-socket\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.785416 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.785251 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/203100f0-5c6d-4372-a64b-f2823d3d7d55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.785416 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.785340 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/203100f0-5c6d-4372-a64b-f2823d3d7d55-crio-socket\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.785666 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.785614 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/203100f0-5c6d-4372-a64b-f2823d3d7d55-data-volume\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.786433 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.786417 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/203100f0-5c6d-4372-a64b-f2823d3d7d55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.787564 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.787545 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/203100f0-5c6d-4372-a64b-f2823d3d7d55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.793652 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.793598 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgnp5\" (UniqueName: \"kubernetes.io/projected/203100f0-5c6d-4372-a64b-f2823d3d7d55-kube-api-access-hgnp5\") pod \"insights-runtime-extractor-4rkbt\" (UID: \"203100f0-5c6d-4372-a64b-f2823d3d7d55\") " pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.836239 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.836200 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4rkbt" Apr 16 18:18:21.980133 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:21.980094 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4rkbt"] Apr 16 18:18:21.984945 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:18:21.984916 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203100f0_5c6d_4372_a64b_f2823d3d7d55.slice/crio-3e2b7507e9d141b277d437b6e403648e6d9bd10440472cffdc6f309f0a4275b5 WatchSource:0}: Error finding container 3e2b7507e9d141b277d437b6e403648e6d9bd10440472cffdc6f309f0a4275b5: Status 404 returned error can't find the container with id 3e2b7507e9d141b277d437b6e403648e6d9bd10440472cffdc6f309f0a4275b5 Apr 16 18:18:22.702585 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:22.702537 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rkbt" event={"ID":"203100f0-5c6d-4372-a64b-f2823d3d7d55","Type":"ContainerStarted","Data":"f3deac9732c34f7f7eba7e33064ef725647a9c8a8b922368d1c1dec0f20acbb1"} Apr 16 18:18:22.702585 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:22.702582 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rkbt" event={"ID":"203100f0-5c6d-4372-a64b-f2823d3d7d55","Type":"ContainerStarted","Data":"406405ce45e5d4d7e4f2a99d1ae00efff6ba12270abc6966abe8d382bab84a83"} Apr 16 18:18:22.702585 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:22.702594 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rkbt" event={"ID":"203100f0-5c6d-4372-a64b-f2823d3d7d55","Type":"ContainerStarted","Data":"3e2b7507e9d141b277d437b6e403648e6d9bd10440472cffdc6f309f0a4275b5"} Apr 16 18:18:24.709720 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:24.709682 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rkbt" event={"ID":"203100f0-5c6d-4372-a64b-f2823d3d7d55","Type":"ContainerStarted","Data":"a2149e01cade9363bfff1c8f4fd01f6700fa2f11c24f64940656126ac77c6ff7"} Apr 16 18:18:24.734444 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:24.734392 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4rkbt" podStartSLOduration=1.84907967 podStartE2EDuration="3.734376529s" podCreationTimestamp="2026-04-16 18:18:21 +0000 UTC" firstStartedPulling="2026-04-16 18:18:22.041170129 +0000 UTC m=+166.434551365" lastFinishedPulling="2026-04-16 18:18:23.926466989 +0000 UTC m=+168.319848224" observedRunningTime="2026-04-16 18:18:24.733178216 +0000 UTC m=+169.126559474" watchObservedRunningTime="2026-04-16 18:18:24.734376529 +0000 UTC m=+169.127757822" Apr 16 18:18:26.315820 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:26.315782 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" podUID="a916945b-958f-4d5d-b8c3-ba3362db924b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Apr 16 18:18:28.209957 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:28.209922 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:18:28.209957 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:28.209952 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:18:28.212390 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:28.212367 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4scgb\"" Apr 16 18:18:28.220977 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:28.220953 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fqnj7" Apr 16 18:18:28.347470 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:28.347440 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fqnj7"] Apr 16 18:18:28.350794 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:18:28.350758 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3ce71d_67a7_431e_9a65_37dd55851b2c.slice/crio-3b8e52e1c8e19ac64e61d3f502105411e2e749fc4c565aaac7ab1eda40887b43 WatchSource:0}: Error finding container 3b8e52e1c8e19ac64e61d3f502105411e2e749fc4c565aaac7ab1eda40887b43: Status 404 returned error can't find the container with id 3b8e52e1c8e19ac64e61d3f502105411e2e749fc4c565aaac7ab1eda40887b43 Apr 16 18:18:28.720615 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:28.720574 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fqnj7" event={"ID":"1f3ce71d-67a7-431e-9a65-37dd55851b2c","Type":"ContainerStarted","Data":"3b8e52e1c8e19ac64e61d3f502105411e2e749fc4c565aaac7ab1eda40887b43"} Apr 16 18:18:29.183880 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.183306 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4b4th"] Apr 16 18:18:29.185858 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.185829 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.188397 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.188310 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Apr 16 18:18:29.188397 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.188367 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Apr 16 18:18:29.188708 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.188688 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g6plr\"" Apr 16 18:18:29.189062 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.188966 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Apr 16 18:18:29.189062 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.189023 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Apr 16 18:18:29.189300 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.189278 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Apr 16 18:18:29.189456 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.189440 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Apr 16 18:18:29.239587 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.239546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.240077 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.239605 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpq67\" (UniqueName: \"kubernetes.io/projected/3f917f86-7a73-4822-9bcf-6f83983701cd-kube-api-access-cpq67\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.240077 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.239656 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f917f86-7a73-4822-9bcf-6f83983701cd-sys\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.240077 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.239702 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-tls\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.240077 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.239916 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f917f86-7a73-4822-9bcf-6f83983701cd-root\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.240077 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.240042 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f917f86-7a73-4822-9bcf-6f83983701cd-metrics-client-ca\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.240322 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.240107 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-textfile\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.240322 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.240154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-wtmp\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.240322 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.240182 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-accelerators-collector-config\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.340850 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.340807 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-tls\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.340850 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.340853 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f917f86-7a73-4822-9bcf-6f83983701cd-root\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.341069 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.340910 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f917f86-7a73-4822-9bcf-6f83983701cd-root\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.341069 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.340969 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f917f86-7a73-4822-9bcf-6f83983701cd-metrics-client-ca\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.341069 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.341027 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-textfile\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.341069 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.341054 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-wtmp\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.341659 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.341082 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-accelerators-collector-config\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.341659 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.341144 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.341659 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.341193 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cpq67\" (UniqueName: \"kubernetes.io/projected/3f917f86-7a73-4822-9bcf-6f83983701cd-kube-api-access-cpq67\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.341659 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.341224 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f917f86-7a73-4822-9bcf-6f83983701cd-sys\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.341659 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.341237 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-wtmp\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.341659 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.341299 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f917f86-7a73-4822-9bcf-6f83983701cd-sys\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.341659 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.341374 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-textfile\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.342109 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.341858 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-accelerators-collector-config\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.342109 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.341978 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f917f86-7a73-4822-9bcf-6f83983701cd-metrics-client-ca\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.343674 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.343652 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-tls\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.344153 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.344124 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f917f86-7a73-4822-9bcf-6f83983701cd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.353133 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.353092 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpq67\" (UniqueName: \"kubernetes.io/projected/3f917f86-7a73-4822-9bcf-6f83983701cd-kube-api-access-cpq67\") pod \"node-exporter-4b4th\" (UID: \"3f917f86-7a73-4822-9bcf-6f83983701cd\") " pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.498209 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:29.498117 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4b4th" Apr 16 18:18:29.854961 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:18:29.854933 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f917f86_7a73_4822_9bcf_6f83983701cd.slice/crio-f7f8eaa1b0fd4bfb58fad7ff7077e9e805c5c1d01912f44b7e88501199832faf WatchSource:0}: Error finding container f7f8eaa1b0fd4bfb58fad7ff7077e9e805c5c1d01912f44b7e88501199832faf: Status 404 returned error can't find the container with id f7f8eaa1b0fd4bfb58fad7ff7077e9e805c5c1d01912f44b7e88501199832faf Apr 16 18:18:30.222501 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.222417 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 16 18:18:30.225153 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.225128 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.227588 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.227560 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Apr 16 18:18:30.227742 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.227591 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Apr 16 18:18:30.227742 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.227610 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Apr 16 18:18:30.227742 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.227560 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Apr 16 18:18:30.227907 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.227738 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sp6hl\"" Apr 16 18:18:30.227907 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.227763 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Apr 16 18:18:30.227907 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.227785 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Apr 16 18:18:30.227907 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.227591 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Apr 16 18:18:30.227907 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.227879 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Apr 16 18:18:30.228157 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.227935 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Apr 16 18:18:30.238365 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.238337 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 16 18:18:30.249149 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249120 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.249615 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvs26\" (UniqueName: \"kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-kube-api-access-xvs26\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.249615 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249202 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.249615 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249235 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.249615 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249272 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.249615 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249356 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.249615 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249394 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.249615 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249423 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.249615 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249451 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.249615 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249477 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.249615 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249588 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-out\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.250044 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-web-config\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.250044 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.249695 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350120 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350085 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350274 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350136 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350274 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350168 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350274 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350191 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350274 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350220 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350274 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350245 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350556 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350312 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-out\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350556 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350336 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-web-config\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350556 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350369 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350556 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350437 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350556 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350462 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xvs26\" (UniqueName: \"kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-kube-api-access-xvs26\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350556 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350514 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.350884 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.350563 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.351405 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.351349 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.351545 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:18:30.351531 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-trusted-ca-bundle podName:1f6e5a0b-63a7-4a08-ac38-5080ba9d645a nodeName:}" failed. No retries permitted until 2026-04-16 18:18:30.851510139 +0000 UTC m=+175.244891394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a") : configmap references non-existent config key: ca-bundle.crt Apr 16 18:18:30.351888 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.351860 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.355956 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.355854 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.355956 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.355911 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.356153 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.356078 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.356236 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.356212 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.356402 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.356370 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-out\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.356652 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.356578 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.356879 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.356855 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-web-config\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.357069 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.357038 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.358725 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.358698 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.360352 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.360308 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvs26\" (UniqueName: \"kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-kube-api-access-xvs26\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.701055 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.701020 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7mtp7" Apr 16 18:18:30.727359 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.727258 2577 generic.go:358] "Generic (PLEG): container finished" podID="3f917f86-7a73-4822-9bcf-6f83983701cd" containerID="e2a690d56912c8e2f1347274738aec9bbea3961e98626ceb32b43e937266e8ee" exitCode=0 Apr 16 18:18:30.727359 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.727326 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4b4th" event={"ID":"3f917f86-7a73-4822-9bcf-6f83983701cd","Type":"ContainerDied","Data":"e2a690d56912c8e2f1347274738aec9bbea3961e98626ceb32b43e937266e8ee"} Apr 16 18:18:30.727569 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.727375 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4b4th" event={"ID":"3f917f86-7a73-4822-9bcf-6f83983701cd","Type":"ContainerStarted","Data":"f7f8eaa1b0fd4bfb58fad7ff7077e9e805c5c1d01912f44b7e88501199832faf"} Apr 16 18:18:30.728963 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.728933 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fqnj7" event={"ID":"1f3ce71d-67a7-431e-9a65-37dd55851b2c","Type":"ContainerStarted","Data":"48af0fa0a1076d902d2fded9202cc217dd7706fb8364bacfe3db115d01fc1592"} Apr 16 18:18:30.771356 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.771308 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fqnj7" podStartSLOduration=139.239311862 podStartE2EDuration="2m20.771286362s" podCreationTimestamp="2026-04-16 18:16:10 +0000 UTC" firstStartedPulling="2026-04-16 18:18:28.352609787 +0000 UTC m=+172.745991022" lastFinishedPulling="2026-04-16 18:18:29.884584286 +0000 UTC m=+174.277965522" observedRunningTime="2026-04-16 18:18:30.765187562 +0000 UTC m=+175.158568823" watchObservedRunningTime="2026-04-16 18:18:30.771286362 +0000 UTC m=+175.164667622" Apr 16 18:18:30.854714 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.854675 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:30.855478 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:30.855453 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:31.135823 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:31.135781 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:18:31.263550 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:31.263514 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 16 18:18:31.266076 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:18:31.266054 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f6e5a0b_63a7_4a08_ac38_5080ba9d645a.slice/crio-4879aaa8f537a1895ee4a544f4a71088ea308cf0cc83adffd8ebc650e3e5a41d WatchSource:0}: Error finding container 4879aaa8f537a1895ee4a544f4a71088ea308cf0cc83adffd8ebc650e3e5a41d: Status 404 returned error can't find the container with id 4879aaa8f537a1895ee4a544f4a71088ea308cf0cc83adffd8ebc650e3e5a41d Apr 16 18:18:31.607823 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:31.607796 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:18:31.734502 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:31.734463 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4b4th" event={"ID":"3f917f86-7a73-4822-9bcf-6f83983701cd","Type":"ContainerStarted","Data":"a47f0b07899e16ccd3414675a2ad1574ebbbb1fc1a1c3ee3ec2ffab8b6c9a61d"} Apr 16 18:18:31.734689 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:31.734509 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4b4th" event={"ID":"3f917f86-7a73-4822-9bcf-6f83983701cd","Type":"ContainerStarted","Data":"798e3a6e20f2bc379bec2b8bc7d4a0de83ab3309d4b09694bdcb82ceafeeed46"} Apr 16 18:18:31.735705 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:31.735677 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerStarted","Data":"4879aaa8f537a1895ee4a544f4a71088ea308cf0cc83adffd8ebc650e3e5a41d"} Apr 16 18:18:31.752902 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:31.752848 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4b4th" podStartSLOduration=2.142082543 podStartE2EDuration="2.752827152s" podCreationTimestamp="2026-04-16 18:18:29 +0000 UTC" firstStartedPulling="2026-04-16 18:18:29.856798696 +0000 UTC m=+174.250179947" lastFinishedPulling="2026-04-16 18:18:30.467543126 +0000 UTC m=+174.860924556" observedRunningTime="2026-04-16 18:18:31.751930805 +0000 UTC m=+176.145312062" watchObservedRunningTime="2026-04-16 18:18:31.752827152 +0000 UTC m=+176.146208410" Apr 16 18:18:32.739802 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:32.739718 2577 generic.go:358] "Generic (PLEG): container finished" podID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerID="95e1910a935efce5a34f49f236b06e5d52391271b7548e23eda6eefb94836ff6" exitCode=0 Apr 16 18:18:32.739802 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:32.739782 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerDied","Data":"95e1910a935efce5a34f49f236b06e5d52391271b7548e23eda6eefb94836ff6"} Apr 16 18:18:33.547244 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.547204 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-579b89d77b-6zfgc"] Apr 16 18:18:33.549540 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.549510 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.552513 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.552488 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-s954r\"" Apr 16 18:18:33.552513 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.552500 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-h2ds2vv5kglo\"" Apr 16 18:18:33.552737 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.552646 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Apr 16 18:18:33.552737 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.552689 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Apr 16 18:18:33.552854 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.552747 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Apr 16 18:18:33.553068 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.553046 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Apr 16 18:18:33.559393 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.559344 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-579b89d77b-6zfgc"] Apr 16 18:18:33.576940 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.576904 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.577125 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.577001 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-secret-metrics-server-tls\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.577125 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.577036 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-client-ca-bundle\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.577125 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.577083 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-metrics-server-audit-profiles\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.577274 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.577163 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-audit-log\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.577274 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.577208 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmjvm\" (UniqueName: \"kubernetes.io/projected/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-kube-api-access-bmjvm\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.577274 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.577268 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-secret-metrics-server-client-certs\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.677887 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.677846 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-audit-log\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.678093 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.677908 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bmjvm\" (UniqueName: \"kubernetes.io/projected/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-kube-api-access-bmjvm\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.678093 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.677967 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-secret-metrics-server-client-certs\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.678217 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.678153 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.678273 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.678221 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-secret-metrics-server-tls\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.678273 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.678260 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-client-ca-bundle\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.678375 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.678294 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-metrics-server-audit-profiles\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.678375 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.678305 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-audit-log\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.679280 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.679249 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.680030 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.680006 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-metrics-server-audit-profiles\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.680923 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.680902 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-secret-metrics-server-client-certs\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.681147 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.681130 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-secret-metrics-server-tls\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.681243 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.681220 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-client-ca-bundle\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.686142 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.686117 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmjvm\" (UniqueName: \"kubernetes.io/projected/11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4-kube-api-access-bmjvm\") pod \"metrics-server-579b89d77b-6zfgc\" (UID: \"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4\") " pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:33.861344 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:33.861252 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:34.079334 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.079278 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-579b89d77b-6zfgc"] Apr 16 18:18:34.082413 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:18:34.082370 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ffe715_7a8a_4c09_94b6_e76e7ed8b6b4.slice/crio-d82cade43524114fe7d920510f33c340589e48ec06aca7b3bfe924ff52a699b1 WatchSource:0}: Error finding container d82cade43524114fe7d920510f33c340589e48ec06aca7b3bfe924ff52a699b1: Status 404 returned error can't find the container with id d82cade43524114fe7d920510f33c340589e48ec06aca7b3bfe924ff52a699b1 Apr 16 18:18:34.209944 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.209916 2577 scope.go:117] "RemoveContainer" containerID="f360c7000c5663ce62b5196f7691bbecbd2e753923e00057d91033476f3448cd" Apr 16 18:18:34.394100 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.394017 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6566964757-bvqbw"] Apr 16 18:18:34.396185 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.396168 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.398462 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.398441 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Apr 16 18:18:34.399036 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.399016 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8dw5z\"" Apr 16 18:18:34.399036 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.399031 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Apr 16 18:18:34.399208 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.399035 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Apr 16 18:18:34.399347 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.399328 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Apr 16 18:18:34.399446 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.399349 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Apr 16 18:18:34.404591 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.404568 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Apr 16 18:18:34.416841 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.416815 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6566964757-bvqbw"] Apr 16 18:18:34.485564 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.485528 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.485764 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.485571 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-federate-client-tls\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.485764 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.485595 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-secret-telemeter-client\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.485764 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.485668 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t249m\" (UniqueName: \"kubernetes.io/projected/1af84565-9bdb-4940-9991-d03274125619-kube-api-access-t249m\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.485764 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.485715 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af84565-9bdb-4940-9991-d03274125619-serving-certs-ca-bundle\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.485764 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.485744 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-telemeter-client-tls\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.485764 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.485764 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1af84565-9bdb-4940-9991-d03274125619-metrics-client-ca\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.485969 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.485804 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af84565-9bdb-4940-9991-d03274125619-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.586873 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.586835 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-federate-client-tls\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.586873 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.586877 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-secret-telemeter-client\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.587108 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.586899 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t249m\" (UniqueName: \"kubernetes.io/projected/1af84565-9bdb-4940-9991-d03274125619-kube-api-access-t249m\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.587108 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.586928 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af84565-9bdb-4940-9991-d03274125619-serving-certs-ca-bundle\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.587108 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.586956 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-telemeter-client-tls\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.587108 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.586985 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1af84565-9bdb-4940-9991-d03274125619-metrics-client-ca\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.587108 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.587026 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af84565-9bdb-4940-9991-d03274125619-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.587338 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.587298 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.587892 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.587841 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af84565-9bdb-4940-9991-d03274125619-serving-certs-ca-bundle\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.588282 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.588256 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af84565-9bdb-4940-9991-d03274125619-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.588607 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.588585 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1af84565-9bdb-4940-9991-d03274125619-metrics-client-ca\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.589733 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.589715 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-secret-telemeter-client\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.589913 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.589887 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.589997 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.589896 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-federate-client-tls\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.590098 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.590079 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1af84565-9bdb-4940-9991-d03274125619-telemeter-client-tls\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.595111 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.595086 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t249m\" (UniqueName: \"kubernetes.io/projected/1af84565-9bdb-4940-9991-d03274125619-kube-api-access-t249m\") pod \"telemeter-client-6566964757-bvqbw\" (UID: \"1af84565-9bdb-4940-9991-d03274125619\") " pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.705981 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.705889 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" Apr 16 18:18:34.749999 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.749962 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/2.log" Apr 16 18:18:34.750205 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.750169 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" event={"ID":"14efee5f-03ef-4047-86f8-ede90b8b5238","Type":"ContainerStarted","Data":"a6ea1641296a412539b1656a2f1ad8e87367237bf440aad8e84f8b44228dcd7c"} Apr 16 18:18:34.751673 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.750803 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:18:34.755075 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.755047 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerStarted","Data":"45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5"} Apr 16 18:18:34.755075 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.755077 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerStarted","Data":"dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00"} Apr 16 18:18:34.755258 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.755087 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerStarted","Data":"58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693"} Apr 16 18:18:34.755258 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.755097 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerStarted","Data":"b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212"} Apr 16 18:18:34.755258 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.755104 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerStarted","Data":"e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74"} Apr 16 18:18:34.757450 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.757420 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" event={"ID":"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4","Type":"ContainerStarted","Data":"d82cade43524114fe7d920510f33c340589e48ec06aca7b3bfe924ff52a699b1"} Apr 16 18:18:34.768335 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.768147 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" podStartSLOduration=43.15153245 podStartE2EDuration="44.768132262s" podCreationTimestamp="2026-04-16 18:17:50 +0000 UTC" firstStartedPulling="2026-04-16 18:17:50.516365036 +0000 UTC m=+134.909746271" lastFinishedPulling="2026-04-16 18:17:52.132964849 +0000 UTC m=+136.526346083" observedRunningTime="2026-04-16 18:18:34.766888203 +0000 UTC m=+179.160269460" watchObservedRunningTime="2026-04-16 18:18:34.768132262 +0000 UTC m=+179.161513518" Apr 16 18:18:34.871920 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:34.871857 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6566964757-bvqbw"] Apr 16 18:18:34.940212 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:18:34.940175 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1af84565_9bdb_4940_9991_d03274125619.slice/crio-7f6ac5140de95a4519ee2ac1c50550b821b039a3e651c8caa46e18a4042facc1 WatchSource:0}: Error finding container 7f6ac5140de95a4519ee2ac1c50550b821b039a3e651c8caa46e18a4042facc1: Status 404 returned error can't find the container with id 7f6ac5140de95a4519ee2ac1c50550b821b039a3e651c8caa46e18a4042facc1 Apr 16 18:18:35.285251 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.285219 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-d87b8d5fc-l9q5d" Apr 16 18:18:35.410427 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.410396 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 16 18:18:35.413711 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.413687 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.417939 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.417899 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Apr 16 18:18:35.418097 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.418029 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Apr 16 18:18:35.418262 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.418243 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Apr 16 18:18:35.418411 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.418364 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Apr 16 18:18:35.418411 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.418380 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Apr 16 18:18:35.418829 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.418745 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Apr 16 18:18:35.418980 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.418959 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Apr 16 18:18:35.419072 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.418977 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Apr 16 18:18:35.419308 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.419282 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-jlpjbromi54k\"" Apr 16 18:18:35.419425 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.419407 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Apr 16 18:18:35.419762 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.419739 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Apr 16 18:18:35.419870 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.419828 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s96f7\"" Apr 16 18:18:35.419935 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.419918 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Apr 16 18:18:35.426070 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.426042 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Apr 16 18:18:35.430546 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.430516 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 16 18:18:35.495313 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495264 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495313 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495314 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495551 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495350 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495551 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495551 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495418 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495551 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495452 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495551 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495477 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495551 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495506 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlkg4\" (UniqueName: \"kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-kube-api-access-xlkg4\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495551 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495532 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config-out\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495903 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495576 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495903 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495647 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495903 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495699 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495903 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495733 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495903 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495761 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495903 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495903 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495835 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495903 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495860 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-web-config\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.495903 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.495899 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.596918 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.596883 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597090 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.596936 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597090 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.596964 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-web-config\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597209 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597093 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597209 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597182 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597211 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597266 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597412 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597304 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597412 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597341 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597412 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597382 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597412 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597407 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597594 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597433 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xlkg4\" (UniqueName: \"kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-kube-api-access-xlkg4\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597594 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597461 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config-out\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597594 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597522 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597594 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597549 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.597594 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597578 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.598040 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597613 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.598040 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597665 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.598040 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.597804 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.598430 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.598401 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.600007 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.599348 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.600165 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.600138 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.600233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.600223 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-web-config\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.601562 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.601503 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.601562 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.601507 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.601877 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.601799 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.602913 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.602892 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.603292 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.603207 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.603292 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.603214 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.603886 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.603852 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.604004 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.603904 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.604349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.604331 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config-out\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.604443 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.604339 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.604800 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.604781 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.605030 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.605014 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.615036 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.615000 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlkg4\" (UniqueName: \"kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-kube-api-access-xlkg4\") pod \"prometheus-k8s-0\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.726731 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.726695 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:35.762177 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.762132 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" event={"ID":"1af84565-9bdb-4940-9991-d03274125619","Type":"ContainerStarted","Data":"7f6ac5140de95a4519ee2ac1c50550b821b039a3e651c8caa46e18a4042facc1"} Apr 16 18:18:35.766611 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.766573 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerStarted","Data":"9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c"} Apr 16 18:18:35.769203 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.769175 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" event={"ID":"11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4","Type":"ContainerStarted","Data":"5785f805b13f786f47ff400b34659661e86c6935c63308c78976e0e17728ac8d"} Apr 16 18:18:35.805355 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.804773 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.108068772 podStartE2EDuration="5.804754077s" podCreationTimestamp="2026-04-16 18:18:30 +0000 UTC" firstStartedPulling="2026-04-16 18:18:31.267933251 +0000 UTC m=+175.661314487" lastFinishedPulling="2026-04-16 18:18:34.964618556 +0000 UTC m=+179.357999792" observedRunningTime="2026-04-16 18:18:35.802890265 +0000 UTC m=+180.196271524" watchObservedRunningTime="2026-04-16 18:18:35.804754077 +0000 UTC m=+180.198135336" Apr 16 18:18:35.821104 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.821055 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" podStartSLOduration=1.345959932 podStartE2EDuration="2.821039606s" podCreationTimestamp="2026-04-16 18:18:33 +0000 UTC" firstStartedPulling="2026-04-16 18:18:34.084440397 +0000 UTC m=+178.477821632" lastFinishedPulling="2026-04-16 18:18:35.559520061 +0000 UTC m=+179.952901306" observedRunningTime="2026-04-16 18:18:35.820476277 +0000 UTC m=+180.213857535" watchObservedRunningTime="2026-04-16 18:18:35.821039606 +0000 UTC m=+180.214420862" Apr 16 18:18:35.896961 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:35.896829 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 16 18:18:35.899405 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:18:35.899374 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod436fc48f_9ecb_4b8d_a8a0_9a7fd7e5428d.slice/crio-ca80dddbcfa9fd5f1186766af34723a42e93b76ca87b298cca55ce00b0b5fe39 WatchSource:0}: Error finding container ca80dddbcfa9fd5f1186766af34723a42e93b76ca87b298cca55ce00b0b5fe39: Status 404 returned error can't find the container with id ca80dddbcfa9fd5f1186766af34723a42e93b76ca87b298cca55ce00b0b5fe39 Apr 16 18:18:36.316729 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:36.316683 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" podUID="a916945b-958f-4d5d-b8c3-ba3362db924b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Apr 16 18:18:36.316902 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:36.316772 2577 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" Apr 16 18:18:36.317470 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:36.317430 2577 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"e735af79f467c23656db0a8087ad1622e57ca4caa2ae9021ab0022f599ec79e6"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" containerMessage="Container service-proxy failed liveness probe, will be restarted" Apr 16 18:18:36.317573 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:36.317495 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" podUID="a916945b-958f-4d5d-b8c3-ba3362db924b" containerName="service-proxy" containerID="cri-o://e735af79f467c23656db0a8087ad1622e57ca4caa2ae9021ab0022f599ec79e6" gracePeriod=30 Apr 16 18:18:36.775300 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:36.775272 2577 generic.go:358] "Generic (PLEG): container finished" podID="a916945b-958f-4d5d-b8c3-ba3362db924b" containerID="e735af79f467c23656db0a8087ad1622e57ca4caa2ae9021ab0022f599ec79e6" exitCode=2 Apr 16 18:18:36.775409 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:36.775344 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" event={"ID":"a916945b-958f-4d5d-b8c3-ba3362db924b","Type":"ContainerDied","Data":"e735af79f467c23656db0a8087ad1622e57ca4caa2ae9021ab0022f599ec79e6"} Apr 16 18:18:36.777024 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:36.776994 2577 generic.go:358] "Generic (PLEG): container finished" podID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerID="8945284704eca41bd9493a77aaed3fb8049c18f3069e32fc092ed018cd3d68c2" exitCode=0 Apr 16 18:18:36.777137 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:36.777104 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerDied","Data":"8945284704eca41bd9493a77aaed3fb8049c18f3069e32fc092ed018cd3d68c2"} Apr 16 18:18:36.777224 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:36.777148 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerStarted","Data":"ca80dddbcfa9fd5f1186766af34723a42e93b76ca87b298cca55ce00b0b5fe39"} Apr 16 18:18:37.783618 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:37.783578 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" event={"ID":"1af84565-9bdb-4940-9991-d03274125619","Type":"ContainerStarted","Data":"f73d11fdfa2e65553919dd6d0f65aaa8eba86e63069f91d7a101849e7804c7e1"} Apr 16 18:18:37.784128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:37.783653 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" event={"ID":"1af84565-9bdb-4940-9991-d03274125619","Type":"ContainerStarted","Data":"d8da713bc50cf148f014474836e56fe2f6020c7cc87156eeb0bb8279ae243ace"} Apr 16 18:18:37.784128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:37.783670 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" event={"ID":"1af84565-9bdb-4940-9991-d03274125619","Type":"ContainerStarted","Data":"23e89a944c9ce3b7d66a48ce432ec1c93db22bbff867ea178238efeb8e370f3a"} Apr 16 18:18:37.786439 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:37.786409 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6847cbfdb5-ww89f" event={"ID":"a916945b-958f-4d5d-b8c3-ba3362db924b","Type":"ContainerStarted","Data":"ad9c1d53095942337d001e45ccecd169c4ed732e887413a825caa6ae04e05f4a"} Apr 16 18:18:37.812107 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:37.812040 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6566964757-bvqbw" podStartSLOduration=1.989730008 podStartE2EDuration="3.812022836s" podCreationTimestamp="2026-04-16 18:18:34 +0000 UTC" firstStartedPulling="2026-04-16 18:18:34.942915504 +0000 UTC m=+179.336296742" lastFinishedPulling="2026-04-16 18:18:36.76520832 +0000 UTC m=+181.158589570" observedRunningTime="2026-04-16 18:18:37.810680258 +0000 UTC m=+182.204061521" watchObservedRunningTime="2026-04-16 18:18:37.812022836 +0000 UTC m=+182.205404094" Apr 16 18:18:39.794316 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:39.794277 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerStarted","Data":"47092abe4c6761fe81ca1f0e3d1bc3e1f59c2c20b4237bdb5e67f8cec3d7df7c"} Apr 16 18:18:39.794316 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:39.794321 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerStarted","Data":"a7ee5d034c30bd7fc9fd4438edf4bc400fed06b3a960f5057f8d5b5fcb54344b"} Apr 16 18:18:41.803609 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:41.803570 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerStarted","Data":"46e2ff4cef73398d273da5dbb080bb5e95b1d423e584e318be97bbb38654927b"} Apr 16 18:18:41.803609 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:41.803607 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerStarted","Data":"7d7d89527e292abc1984020be7e7010458534274fc54a5705cd31fe517e6d37f"} Apr 16 18:18:41.803609 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:41.803616 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerStarted","Data":"6909a90e4cdb4f58b19c2d1cf549a441019ec6260182316fa793cf8a15c30ac0"} Apr 16 18:18:41.804229 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:41.803657 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerStarted","Data":"ee969103c0175c78a4404c94f6cbd66cd30c05a5f3940688259b9c2094642368"} Apr 16 18:18:41.832259 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:41.832184 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.49968712 podStartE2EDuration="6.832170049s" podCreationTimestamp="2026-04-16 18:18:35 +0000 UTC" firstStartedPulling="2026-04-16 18:18:36.77855795 +0000 UTC m=+181.171939185" lastFinishedPulling="2026-04-16 18:18:41.111040879 +0000 UTC m=+185.504422114" observedRunningTime="2026-04-16 18:18:41.831135839 +0000 UTC m=+186.224517097" watchObservedRunningTime="2026-04-16 18:18:41.832170049 +0000 UTC m=+186.225551306" Apr 16 18:18:45.727547 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:45.727508 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:18:46.621017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.620980 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" podUID="2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" containerName="registry" containerID="cri-o://8f6960f6e1ef603acc782bd4e49541c06efe96df274337834ecff2820b8bfa42" gracePeriod=30 Apr 16 18:18:46.819184 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.819151 2577 generic.go:358] "Generic (PLEG): container finished" podID="2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" containerID="8f6960f6e1ef603acc782bd4e49541c06efe96df274337834ecff2820b8bfa42" exitCode=0 Apr 16 18:18:46.819535 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.819200 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" event={"ID":"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94","Type":"ContainerDied","Data":"8f6960f6e1ef603acc782bd4e49541c06efe96df274337834ecff2820b8bfa42"} Apr 16 18:18:46.862776 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.862750 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:18:46.907749 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.907651 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-image-registry-private-configuration\") pod \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " Apr 16 18:18:46.907749 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.907720 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-trusted-ca\") pod \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " Apr 16 18:18:46.908005 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.907763 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-installation-pull-secrets\") pod \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " Apr 16 18:18:46.908005 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.907788 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls\") pod \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " Apr 16 18:18:46.908005 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.907834 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtd5v\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-kube-api-access-gtd5v\") pod \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " Apr 16 18:18:46.908005 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.907854 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-certificates\") pod \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " Apr 16 18:18:46.908005 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.907889 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-bound-sa-token\") pod \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " Apr 16 18:18:46.908005 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.907917 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-ca-trust-extracted\") pod \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\" (UID: \"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94\") " Apr 16 18:18:46.908289 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.908215 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 18:18:46.908676 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.908615 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 18:18:46.910704 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.910655 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 18:18:46.910704 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.910674 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:18:46.910871 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.910786 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 18:18:46.910871 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.910797 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:18:46.910871 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.910835 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-kube-api-access-gtd5v" (OuterVolumeSpecName: "kube-api-access-gtd5v") pod "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94"). InnerVolumeSpecName "kube-api-access-gtd5v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 18:18:46.916074 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:46.916044 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" (UID: "2139e7d8-f98a-49e0-a5a4-1d715b1a9a94"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 18:18:47.009490 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.009455 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-installation-pull-secrets\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:18:47.009490 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.009486 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-tls\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:18:47.009490 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.009495 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtd5v\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-kube-api-access-gtd5v\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:18:47.009771 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.009506 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-registry-certificates\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:18:47.009771 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.009515 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-bound-sa-token\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:18:47.009771 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.009524 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-ca-trust-extracted\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:18:47.009771 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.009535 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-image-registry-private-configuration\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:18:47.009771 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.009545 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94-trusted-ca\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:18:47.824830 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.824793 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" event={"ID":"2139e7d8-f98a-49e0-a5a4-1d715b1a9a94","Type":"ContainerDied","Data":"733434ce9c863b5c6d158eaf1348d9c2db6981ebb396ad002d52e4d0528f7258"} Apr 16 18:18:47.825424 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.825406 2577 scope.go:117] "RemoveContainer" containerID="8f6960f6e1ef603acc782bd4e49541c06efe96df274337834ecff2820b8bfa42" Apr 16 18:18:47.825658 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.825439 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79699d8d5-9zpdb" Apr 16 18:18:47.857710 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.857679 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79699d8d5-9zpdb"] Apr 16 18:18:47.862171 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:47.862147 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-79699d8d5-9zpdb"] Apr 16 18:18:48.213834 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:48.213752 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" path="/var/lib/kubelet/pods/2139e7d8-f98a-49e0-a5a4-1d715b1a9a94/volumes" Apr 16 18:18:53.861742 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:53.861700 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:18:53.862152 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:18:53.861790 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:19:00.784301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:00.784272 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fqnj7_1f3ce71d-67a7-431e-9a65-37dd55851b2c/serve-healthcheck-canary/0.log" Apr 16 18:19:13.872351 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:13.872310 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:19:13.877233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:13.877204 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-579b89d77b-6zfgc" Apr 16 18:19:28.956957 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:28.956918 2577 generic.go:358] "Generic (PLEG): container finished" podID="df11f2db-bfe9-48ff-a2f7-cfc24826b010" containerID="1500b8d59934c62ee1846ba19cfdf0faec68aa6de9607a89460d8e0d577dc4cc" exitCode=0 Apr 16 18:19:28.957312 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:28.956965 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" event={"ID":"df11f2db-bfe9-48ff-a2f7-cfc24826b010","Type":"ContainerDied","Data":"1500b8d59934c62ee1846ba19cfdf0faec68aa6de9607a89460d8e0d577dc4cc"} Apr 16 18:19:28.957312 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:28.957273 2577 scope.go:117] "RemoveContainer" containerID="1500b8d59934c62ee1846ba19cfdf0faec68aa6de9607a89460d8e0d577dc4cc" Apr 16 18:19:29.961457 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:29.961416 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-6sb7r" event={"ID":"df11f2db-bfe9-48ff-a2f7-cfc24826b010","Type":"ContainerStarted","Data":"5457f2276cb83527dd58671838fe92ae2287869d8d15dc54a2e5282466b82f22"} Apr 16 18:19:35.727218 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:35.727175 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:35.748277 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:35.748248 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:35.994890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:35.994793 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:47.958537 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:47.958485 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:19:47.961082 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:47.961062 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5609e075-d6f5-4642-a711-20f608288876-metrics-certs\") pod \"network-metrics-daemon-lp8x6\" (UID: \"5609e075-d6f5-4642-a711-20f608288876\") " pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:19:48.012995 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:48.012743 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4plx6\"" Apr 16 18:19:48.020610 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:48.020572 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lp8x6" Apr 16 18:19:48.153128 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:48.153086 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lp8x6"] Apr 16 18:19:48.156757 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:19:48.156728 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5609e075_d6f5_4642_a711_20f608288876.slice/crio-50b4fddfa4f8d98c4341f44c3f30703b412bbaab4aef53814199019830d15f5e WatchSource:0}: Error finding container 50b4fddfa4f8d98c4341f44c3f30703b412bbaab4aef53814199019830d15f5e: Status 404 returned error can't find the container with id 50b4fddfa4f8d98c4341f44c3f30703b412bbaab4aef53814199019830d15f5e Apr 16 18:19:49.016872 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:49.016830 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lp8x6" event={"ID":"5609e075-d6f5-4642-a711-20f608288876","Type":"ContainerStarted","Data":"50b4fddfa4f8d98c4341f44c3f30703b412bbaab4aef53814199019830d15f5e"} Apr 16 18:19:49.564879 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:49.564788 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 16 18:19:49.565424 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:49.565394 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="alertmanager" containerID="cri-o://e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74" gracePeriod=120 Apr 16 18:19:49.565500 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:49.565467 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="prom-label-proxy" containerID="cri-o://9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c" gracePeriod=120 Apr 16 18:19:49.565556 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:49.565501 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="config-reloader" containerID="cri-o://b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212" gracePeriod=120 Apr 16 18:19:49.565556 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:49.565492 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy-web" containerID="cri-o://58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693" gracePeriod=120 Apr 16 18:19:49.565556 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:49.565518 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy" containerID="cri-o://dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00" gracePeriod=120 Apr 16 18:19:49.565736 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:49.565443 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy-metric" containerID="cri-o://45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5" gracePeriod=120 Apr 16 18:19:50.023094 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.023055 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lp8x6" event={"ID":"5609e075-d6f5-4642-a711-20f608288876","Type":"ContainerStarted","Data":"bbfff79ad1daaa933c5e30f7215444d4af262199a4d82424bc0a089a0ffe0aa1"} Apr 16 18:19:50.023094 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.023097 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lp8x6" event={"ID":"5609e075-d6f5-4642-a711-20f608288876","Type":"ContainerStarted","Data":"fecce4f20f026ceb3ca484bd18f51a61c2f38676170c6eb998ba80879e5c739b"} Apr 16 18:19:50.026197 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.026169 2577 generic.go:358] "Generic (PLEG): container finished" podID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerID="9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c" exitCode=0 Apr 16 18:19:50.026344 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.026192 2577 generic.go:358] "Generic (PLEG): container finished" podID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerID="45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5" exitCode=0 Apr 16 18:19:50.026344 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.026219 2577 generic.go:358] "Generic (PLEG): container finished" podID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerID="dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00" exitCode=0 Apr 16 18:19:50.026344 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.026225 2577 generic.go:358] "Generic (PLEG): container finished" podID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerID="b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212" exitCode=0 Apr 16 18:19:50.026344 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.026230 2577 generic.go:358] "Generic (PLEG): container finished" podID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerID="e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74" exitCode=0 Apr 16 18:19:50.026344 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.026242 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerDied","Data":"9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c"} Apr 16 18:19:50.026344 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.026278 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerDied","Data":"45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5"} Apr 16 18:19:50.026344 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.026291 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerDied","Data":"dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00"} Apr 16 18:19:50.026344 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.026303 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerDied","Data":"b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212"} Apr 16 18:19:50.026344 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.026316 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerDied","Data":"e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74"} Apr 16 18:19:50.039607 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.039558 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lp8x6" podStartSLOduration=252.986239834 podStartE2EDuration="4m14.039538209s" podCreationTimestamp="2026-04-16 18:15:36 +0000 UTC" firstStartedPulling="2026-04-16 18:19:48.158578593 +0000 UTC m=+252.551959828" lastFinishedPulling="2026-04-16 18:19:49.211876956 +0000 UTC m=+253.605258203" observedRunningTime="2026-04-16 18:19:50.039334876 +0000 UTC m=+254.432716132" watchObservedRunningTime="2026-04-16 18:19:50.039538209 +0000 UTC m=+254.432919466" Apr 16 18:19:50.809419 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.809394 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:50.986469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986385 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-main-tls\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.986469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986424 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-tls-assets\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.986469 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986463 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvs26\" (UniqueName: \"kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-kube-api-access-xvs26\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.986750 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986490 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-out\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.986750 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986513 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-main-db\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.986750 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986544 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-web-config\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.986750 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986584 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-cluster-tls-config\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.986750 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986680 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.986750 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986705 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-volume\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.987017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986750 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-trusted-ca-bundle\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.987017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986803 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.987017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986829 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-metrics-client-ca\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.987017 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.986861 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-web\") pod \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\" (UID: \"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a\") " Apr 16 18:19:50.988050 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.987950 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 18:19:50.988183 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.988048 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 18:19:50.988183 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.988055 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 18:19:50.990515 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.990478 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:50.991464 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.991434 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-out" (OuterVolumeSpecName: "config-out") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 18:19:50.991583 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.991455 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:50.991583 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.991495 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 18:19:50.991855 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.991822 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-volume" (OuterVolumeSpecName: "config-volume") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:50.991855 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.991840 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:50.992147 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.992119 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-kube-api-access-xvs26" (OuterVolumeSpecName: "kube-api-access-xvs26") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "kube-api-access-xvs26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 18:19:50.992607 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.992576 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:50.995186 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:50.995063 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:51.002486 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.002455 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-web-config" (OuterVolumeSpecName: "web-config") pod "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" (UID: "1f6e5a0b-63a7-4a08-ac38-5080ba9d645a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:51.032322 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.032287 2577 generic.go:358] "Generic (PLEG): container finished" podID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerID="58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693" exitCode=0 Apr 16 18:19:51.032774 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.032379 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerDied","Data":"58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693"} Apr 16 18:19:51.032774 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.032404 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.032774 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.032419 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f6e5a0b-63a7-4a08-ac38-5080ba9d645a","Type":"ContainerDied","Data":"4879aaa8f537a1895ee4a544f4a71088ea308cf0cc83adffd8ebc650e3e5a41d"} Apr 16 18:19:51.032774 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.032437 2577 scope.go:117] "RemoveContainer" containerID="9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c" Apr 16 18:19:51.040502 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.040480 2577 scope.go:117] "RemoveContainer" containerID="45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5" Apr 16 18:19:51.048003 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.047979 2577 scope.go:117] "RemoveContainer" containerID="dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00" Apr 16 18:19:51.054816 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.054789 2577 scope.go:117] "RemoveContainer" containerID="58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693" Apr 16 18:19:51.057877 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.057849 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 16 18:19:51.061592 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.061567 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 16 18:19:51.062944 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.062926 2577 scope.go:117] "RemoveContainer" containerID="b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212" Apr 16 18:19:51.070411 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.070389 2577 scope.go:117] "RemoveContainer" containerID="e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74" Apr 16 18:19:51.077556 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.077529 2577 scope.go:117] "RemoveContainer" containerID="95e1910a935efce5a34f49f236b06e5d52391271b7548e23eda6eefb94836ff6" Apr 16 18:19:51.085226 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.085209 2577 scope.go:117] "RemoveContainer" containerID="9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c" Apr 16 18:19:51.085524 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:19:51.085503 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c\": container with ID starting with 9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c not found: ID does not exist" containerID="9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c" Apr 16 18:19:51.085596 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.085532 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c"} err="failed to get container status \"9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c\": rpc error: code = NotFound desc = could not find container \"9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c\": container with ID starting with 9f69e02647cf8b9f8bd3fefc5fabed7d20515cf6226297612a962addcd54868c not found: ID does not exist" Apr 16 18:19:51.085596 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.085552 2577 scope.go:117] "RemoveContainer" containerID="45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5" Apr 16 18:19:51.085838 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:19:51.085815 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5\": container with ID starting with 45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5 not found: ID does not exist" containerID="45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5" Apr 16 18:19:51.085882 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.085848 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5"} err="failed to get container status \"45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5\": rpc error: code = NotFound desc = could not find container \"45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5\": container with ID starting with 45bad9d7cdbcfc74d6066c3d70dc92ac78c541dc890ace96ff4738a7899b2ab5 not found: ID does not exist" Apr 16 18:19:51.085882 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.085867 2577 scope.go:117] "RemoveContainer" containerID="dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00" Apr 16 18:19:51.086111 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:19:51.086092 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00\": container with ID starting with dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00 not found: ID does not exist" containerID="dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00" Apr 16 18:19:51.086165 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.086114 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00"} err="failed to get container status \"dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00\": rpc error: code = NotFound desc = could not find container \"dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00\": container with ID starting with dd30d3ec963a24836cd2651b67ecf69650c57f4ba93f03a0dd8c90bbdf8fcd00 not found: ID does not exist" Apr 16 18:19:51.086165 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.086127 2577 scope.go:117] "RemoveContainer" containerID="58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693" Apr 16 18:19:51.086309 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:19:51.086291 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693\": container with ID starting with 58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693 not found: ID does not exist" containerID="58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693" Apr 16 18:19:51.086352 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.086315 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693"} err="failed to get container status \"58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693\": rpc error: code = NotFound desc = could not find container \"58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693\": container with ID starting with 58efa724b3a80125beaca4ff443115ad1dd1859c86d163048663f43d4705c693 not found: ID does not exist" Apr 16 18:19:51.086352 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.086329 2577 scope.go:117] "RemoveContainer" containerID="b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212" Apr 16 18:19:51.086540 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:19:51.086525 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212\": container with ID starting with b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212 not found: ID does not exist" containerID="b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212" Apr 16 18:19:51.086576 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.086543 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212"} err="failed to get container status \"b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212\": rpc error: code = NotFound desc = could not find container \"b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212\": container with ID starting with b81358966632219b66deedae9de4a518fe44214b0860db4d5a0efa53918ee212 not found: ID does not exist" Apr 16 18:19:51.086576 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.086555 2577 scope.go:117] "RemoveContainer" containerID="e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74" Apr 16 18:19:51.086864 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:19:51.086849 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74\": container with ID starting with e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74 not found: ID does not exist" containerID="e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74" Apr 16 18:19:51.086930 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.086866 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74"} err="failed to get container status \"e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74\": rpc error: code = NotFound desc = could not find container \"e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74\": container with ID starting with e7e8f9b51f0d550a27c61b4aa289c6ec427d82297e9365aec41005ea0123fb74 not found: ID does not exist" Apr 16 18:19:51.086930 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.086889 2577 scope.go:117] "RemoveContainer" containerID="95e1910a935efce5a34f49f236b06e5d52391271b7548e23eda6eefb94836ff6" Apr 16 18:19:51.087134 ip-10-0-138-22 kubenswrapper[2577]: E0416 18:19:51.087118 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95e1910a935efce5a34f49f236b06e5d52391271b7548e23eda6eefb94836ff6\": container with ID starting with 95e1910a935efce5a34f49f236b06e5d52391271b7548e23eda6eefb94836ff6 not found: ID does not exist" containerID="95e1910a935efce5a34f49f236b06e5d52391271b7548e23eda6eefb94836ff6" Apr 16 18:19:51.087176 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087140 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95e1910a935efce5a34f49f236b06e5d52391271b7548e23eda6eefb94836ff6"} err="failed to get container status \"95e1910a935efce5a34f49f236b06e5d52391271b7548e23eda6eefb94836ff6\": rpc error: code = NotFound desc = could not find container \"95e1910a935efce5a34f49f236b06e5d52391271b7548e23eda6eefb94836ff6\": container with ID starting with 95e1910a935efce5a34f49f236b06e5d52391271b7548e23eda6eefb94836ff6 not found: ID does not exist" Apr 16 18:19:51.087438 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087425 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvs26\" (UniqueName: \"kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-kube-api-access-xvs26\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087474 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087443 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-out\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087474 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087452 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-main-db\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087474 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087462 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-web-config\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087474 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087471 2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-cluster-tls-config\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087588 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087480 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087588 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087489 2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-config-volume\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087588 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087497 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087588 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087506 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087588 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087515 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-metrics-client-ca\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087588 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087524 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087588 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087533 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-secret-alertmanager-main-tls\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.087588 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.087542 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a-tls-assets\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:51.090111 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090089 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 16 18:19:51.090383 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090369 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" containerName="registry" Apr 16 18:19:51.090462 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090388 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" containerName="registry" Apr 16 18:19:51.090462 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090404 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="alertmanager" Apr 16 18:19:51.090462 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090413 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="alertmanager" Apr 16 18:19:51.090462 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090424 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy-web" Apr 16 18:19:51.090462 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090433 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy-web" Apr 16 18:19:51.090462 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090444 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy" Apr 16 18:19:51.090462 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090452 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090470 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="config-reloader" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090478 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="config-reloader" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090505 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy-metric" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090513 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy-metric" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090524 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="prom-label-proxy" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090533 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="prom-label-proxy" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090545 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="init-config-reloader" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090554 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="init-config-reloader" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090650 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="prom-label-proxy" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090665 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="2139e7d8-f98a-49e0-a5a4-1d715b1a9a94" containerName="registry" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090674 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy-web" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090684 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="config-reloader" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090695 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090706 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="kube-rbac-proxy-metric" Apr 16 18:19:51.090809 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.090717 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" containerName="alertmanager" Apr 16 18:19:51.095922 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.095901 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.098233 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.098212 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Apr 16 18:19:51.098371 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.098257 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Apr 16 18:19:51.098371 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.098329 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Apr 16 18:19:51.098488 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.098473 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Apr 16 18:19:51.098810 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.098791 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Apr 16 18:19:51.098901 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.098824 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Apr 16 18:19:51.099216 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.099194 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Apr 16 18:19:51.099326 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.099201 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Apr 16 18:19:51.099714 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.099700 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sp6hl\"" Apr 16 18:19:51.104489 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.104467 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Apr 16 18:19:51.117936 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.117898 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 16 18:19:51.188684 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.188608 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.188886 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.188720 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/462b34a0-4db7-486a-9db0-bc0614828026-tls-assets\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.188886 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.188751 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.188886 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.188859 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.189058 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.188890 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/462b34a0-4db7-486a-9db0-bc0614828026-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.189058 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.188915 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzhsm\" (UniqueName: \"kubernetes.io/projected/462b34a0-4db7-486a-9db0-bc0614828026-kube-api-access-fzhsm\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.189058 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.188967 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.189058 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.188999 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-web-config\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.189058 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.189014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.189058 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.189043 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/462b34a0-4db7-486a-9db0-bc0614828026-config-out\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.189267 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.189107 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-config-volume\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.189267 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.189136 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/462b34a0-4db7-486a-9db0-bc0614828026-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.189267 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.189154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/462b34a0-4db7-486a-9db0-bc0614828026-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.289958 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.289908 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.289958 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.289966 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-web-config\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.290211 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.289984 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.290211 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.290112 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/462b34a0-4db7-486a-9db0-bc0614828026-config-out\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.290211 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.290167 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-config-volume\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.290211 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.290193 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/462b34a0-4db7-486a-9db0-bc0614828026-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.290395 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.290218 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/462b34a0-4db7-486a-9db0-bc0614828026-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.290395 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.290256 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.290395 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.290299 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/462b34a0-4db7-486a-9db0-bc0614828026-tls-assets\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.290395 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.290332 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.290395 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.290390 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.290661 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.290416 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/462b34a0-4db7-486a-9db0-bc0614828026-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.290661 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.290442 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fzhsm\" (UniqueName: \"kubernetes.io/projected/462b34a0-4db7-486a-9db0-bc0614828026-kube-api-access-fzhsm\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.291086 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.291059 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/462b34a0-4db7-486a-9db0-bc0614828026-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.291167 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.291131 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/462b34a0-4db7-486a-9db0-bc0614828026-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.293682 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.293239 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/462b34a0-4db7-486a-9db0-bc0614828026-config-out\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.293682 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.293359 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-web-config\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.293682 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.293359 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.293682 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.293499 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/462b34a0-4db7-486a-9db0-bc0614828026-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.293682 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.293599 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.293952 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.293696 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.293952 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.293768 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-config-volume\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.294052 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.294034 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.294086 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.294064 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/462b34a0-4db7-486a-9db0-bc0614828026-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.295345 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.295330 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/462b34a0-4db7-486a-9db0-bc0614828026-tls-assets\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.299216 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.299198 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzhsm\" (UniqueName: \"kubernetes.io/projected/462b34a0-4db7-486a-9db0-bc0614828026-kube-api-access-fzhsm\") pod \"alertmanager-main-0\" (UID: \"462b34a0-4db7-486a-9db0-bc0614828026\") " pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.406546 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.406502 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 16 18:19:51.550140 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:51.550029 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 16 18:19:51.553824 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:19:51.553793 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod462b34a0_4db7_486a_9db0_bc0614828026.slice/crio-574731d6494139556731342614133a17accb36958a84c90a728f34969b31b026 WatchSource:0}: Error finding container 574731d6494139556731342614133a17accb36958a84c90a728f34969b31b026: Status 404 returned error can't find the container with id 574731d6494139556731342614133a17accb36958a84c90a728f34969b31b026 Apr 16 18:19:52.040050 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:52.040016 2577 generic.go:358] "Generic (PLEG): container finished" podID="462b34a0-4db7-486a-9db0-bc0614828026" containerID="008c1d2b787c352372d4331738636973c1da0dd60711ad6eae91ae46992003c3" exitCode=0 Apr 16 18:19:52.040440 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:52.040106 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"462b34a0-4db7-486a-9db0-bc0614828026","Type":"ContainerDied","Data":"008c1d2b787c352372d4331738636973c1da0dd60711ad6eae91ae46992003c3"} Apr 16 18:19:52.040440 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:52.040141 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"462b34a0-4db7-486a-9db0-bc0614828026","Type":"ContainerStarted","Data":"574731d6494139556731342614133a17accb36958a84c90a728f34969b31b026"} Apr 16 18:19:52.215540 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:52.215505 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f6e5a0b-63a7-4a08-ac38-5080ba9d645a" path="/var/lib/kubelet/pods/1f6e5a0b-63a7-4a08-ac38-5080ba9d645a/volumes" Apr 16 18:19:53.047139 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.047103 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"462b34a0-4db7-486a-9db0-bc0614828026","Type":"ContainerStarted","Data":"0880f6dca128031166aab40b4dc105eeb2bedcfa877471f8b1fa2eaf6be96e6e"} Apr 16 18:19:53.047139 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.047139 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"462b34a0-4db7-486a-9db0-bc0614828026","Type":"ContainerStarted","Data":"54e5b16f9bcbfba4d6b7910cdf4061a5c17084a3fbc61553762f6fd1c62c748e"} Apr 16 18:19:53.047651 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.047153 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"462b34a0-4db7-486a-9db0-bc0614828026","Type":"ContainerStarted","Data":"7af09a89853a3eb63312db7d38a7ad5d3a0bc7f4bbe202e1280545bb6fbb3550"} Apr 16 18:19:53.047651 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.047165 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"462b34a0-4db7-486a-9db0-bc0614828026","Type":"ContainerStarted","Data":"e9aa07f798669ab07ae44ab7826a21a60a538515f1e1981b2ae2af79abf24214"} Apr 16 18:19:53.047651 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.047174 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"462b34a0-4db7-486a-9db0-bc0614828026","Type":"ContainerStarted","Data":"e8fe487150561f432adb84c6f02508205cd2cb1279bcc43a1bb9bd78b1e9e689"} Apr 16 18:19:53.047651 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.047184 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"462b34a0-4db7-486a-9db0-bc0614828026","Type":"ContainerStarted","Data":"95d62f022fe51d10e78d8de42a304c65f88d4b32ceb570c9e790b4f3f80d9e0e"} Apr 16 18:19:53.072787 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.072739 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.072716005 podStartE2EDuration="2.072716005s" podCreationTimestamp="2026-04-16 18:19:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 18:19:53.071486036 +0000 UTC m=+257.464867284" watchObservedRunningTime="2026-04-16 18:19:53.072716005 +0000 UTC m=+257.466097264" Apr 16 18:19:53.843851 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.843811 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 16 18:19:53.844394 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.844335 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="prometheus" containerID="cri-o://a7ee5d034c30bd7fc9fd4438edf4bc400fed06b3a960f5057f8d5b5fcb54344b" gracePeriod=600 Apr 16 18:19:53.844520 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.844395 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy-web" containerID="cri-o://6909a90e4cdb4f58b19c2d1cf549a441019ec6260182316fa793cf8a15c30ac0" gracePeriod=600 Apr 16 18:19:53.844520 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.844379 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="thanos-sidecar" containerID="cri-o://ee969103c0175c78a4404c94f6cbd66cd30c05a5f3940688259b9c2094642368" gracePeriod=600 Apr 16 18:19:53.844520 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.844423 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="config-reloader" containerID="cri-o://47092abe4c6761fe81ca1f0e3d1bc3e1f59c2c20b4237bdb5e67f8cec3d7df7c" gracePeriod=600 Apr 16 18:19:53.844520 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.844365 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy" containerID="cri-o://7d7d89527e292abc1984020be7e7010458534274fc54a5705cd31fe517e6d37f" gracePeriod=600 Apr 16 18:19:53.845660 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:53.844797 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://46e2ff4cef73398d273da5dbb080bb5e95b1d423e584e318be97bbb38654927b" gracePeriod=600 Apr 16 18:19:54.054351 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054316 2577 generic.go:358] "Generic (PLEG): container finished" podID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerID="46e2ff4cef73398d273da5dbb080bb5e95b1d423e584e318be97bbb38654927b" exitCode=0 Apr 16 18:19:54.054351 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054345 2577 generic.go:358] "Generic (PLEG): container finished" podID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerID="7d7d89527e292abc1984020be7e7010458534274fc54a5705cd31fe517e6d37f" exitCode=0 Apr 16 18:19:54.054351 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054351 2577 generic.go:358] "Generic (PLEG): container finished" podID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerID="6909a90e4cdb4f58b19c2d1cf549a441019ec6260182316fa793cf8a15c30ac0" exitCode=0 Apr 16 18:19:54.054351 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054357 2577 generic.go:358] "Generic (PLEG): container finished" podID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerID="ee969103c0175c78a4404c94f6cbd66cd30c05a5f3940688259b9c2094642368" exitCode=0 Apr 16 18:19:54.054875 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054367 2577 generic.go:358] "Generic (PLEG): container finished" podID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerID="47092abe4c6761fe81ca1f0e3d1bc3e1f59c2c20b4237bdb5e67f8cec3d7df7c" exitCode=0 Apr 16 18:19:54.054875 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054377 2577 generic.go:358] "Generic (PLEG): container finished" podID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerID="a7ee5d034c30bd7fc9fd4438edf4bc400fed06b3a960f5057f8d5b5fcb54344b" exitCode=0 Apr 16 18:19:54.054875 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054395 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerDied","Data":"46e2ff4cef73398d273da5dbb080bb5e95b1d423e584e318be97bbb38654927b"} Apr 16 18:19:54.054875 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054441 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerDied","Data":"7d7d89527e292abc1984020be7e7010458534274fc54a5705cd31fe517e6d37f"} Apr 16 18:19:54.054875 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054457 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerDied","Data":"6909a90e4cdb4f58b19c2d1cf549a441019ec6260182316fa793cf8a15c30ac0"} Apr 16 18:19:54.054875 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054471 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerDied","Data":"ee969103c0175c78a4404c94f6cbd66cd30c05a5f3940688259b9c2094642368"} Apr 16 18:19:54.054875 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054486 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerDied","Data":"47092abe4c6761fe81ca1f0e3d1bc3e1f59c2c20b4237bdb5e67f8cec3d7df7c"} Apr 16 18:19:54.054875 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.054501 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerDied","Data":"a7ee5d034c30bd7fc9fd4438edf4bc400fed06b3a960f5057f8d5b5fcb54344b"} Apr 16 18:19:54.090289 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.090263 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:54.215610 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215532 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-kubelet-serving-ca-bundle\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215610 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215572 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-grpc-tls\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215610 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215590 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-serving-certs-ca-bundle\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215891 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215639 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-db\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215891 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215681 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215891 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215699 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-tls-assets\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215891 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215714 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215891 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215747 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-trusted-ca-bundle\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215891 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215788 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215891 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215810 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config-out\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215891 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215839 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-rulefiles-0\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215891 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215863 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-metrics-client-ca\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.215891 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215888 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-web-config\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.216345 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215914 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-metrics-client-certs\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.216345 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215946 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-thanos-prometheus-http-client-file\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.216345 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.215998 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlkg4\" (UniqueName: \"kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-kube-api-access-xlkg4\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.216345 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.216030 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-kube-rbac-proxy\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.216345 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.216022 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 18:19:54.216345 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.216045 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 18:19:54.216345 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.216061 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-tls\") pod \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\" (UID: \"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d\") " Apr 16 18:19:54.216711 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.216599 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.216711 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.216621 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.218050 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.217736 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 18:19:54.219064 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.218992 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:54.219064 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.219027 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:54.219306 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.219230 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 18:19:54.219607 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.219418 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 18:19:54.219607 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.219561 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 18:19:54.219780 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.219671 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:54.219780 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.219753 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:54.219952 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.219913 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 18:19:54.220706 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.220601 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config" (OuterVolumeSpecName: "config") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:54.220835 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.220814 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:54.221235 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.221214 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config-out" (OuterVolumeSpecName: "config-out") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 18:19:54.221540 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.221522 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-kube-api-access-xlkg4" (OuterVolumeSpecName: "kube-api-access-xlkg4") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "kube-api-access-xlkg4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 18:19:54.221679 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.221663 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:54.221815 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.221795 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:54.231117 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.231084 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-web-config" (OuterVolumeSpecName: "web-config") pod "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" (UID: "436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 18:19:54.318087 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318051 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlkg4\" (UniqueName: \"kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-kube-api-access-xlkg4\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318087 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318083 2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-kube-rbac-proxy\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318098 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318109 2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-grpc-tls\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318118 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-db\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318129 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318138 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-tls-assets\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318149 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318165 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318179 2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318192 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-config-out\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318205 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318218 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-configmap-metrics-client-ca\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318230 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-web-config\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318245 2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-secret-metrics-client-certs\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:54.318301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:54.318259 2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-22.ec2.internal\" DevicePath \"\"" Apr 16 18:19:55.060053 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.060012 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d","Type":"ContainerDied","Data":"ca80dddbcfa9fd5f1186766af34723a42e93b76ca87b298cca55ce00b0b5fe39"} Apr 16 18:19:55.060053 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.060048 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.060497 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.060068 2577 scope.go:117] "RemoveContainer" containerID="46e2ff4cef73398d273da5dbb080bb5e95b1d423e584e318be97bbb38654927b" Apr 16 18:19:55.067744 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.067726 2577 scope.go:117] "RemoveContainer" containerID="7d7d89527e292abc1984020be7e7010458534274fc54a5705cd31fe517e6d37f" Apr 16 18:19:55.075213 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.075193 2577 scope.go:117] "RemoveContainer" containerID="6909a90e4cdb4f58b19c2d1cf549a441019ec6260182316fa793cf8a15c30ac0" Apr 16 18:19:55.082722 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.082581 2577 scope.go:117] "RemoveContainer" containerID="ee969103c0175c78a4404c94f6cbd66cd30c05a5f3940688259b9c2094642368" Apr 16 18:19:55.083982 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.083862 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 16 18:19:55.088506 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.088479 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 16 18:19:55.091069 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.091046 2577 scope.go:117] "RemoveContainer" containerID="47092abe4c6761fe81ca1f0e3d1bc3e1f59c2c20b4237bdb5e67f8cec3d7df7c" Apr 16 18:19:55.098351 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.098330 2577 scope.go:117] "RemoveContainer" containerID="a7ee5d034c30bd7fc9fd4438edf4bc400fed06b3a960f5057f8d5b5fcb54344b" Apr 16 18:19:55.106004 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.105984 2577 scope.go:117] "RemoveContainer" containerID="8945284704eca41bd9493a77aaed3fb8049c18f3069e32fc092ed018cd3d68c2" Apr 16 18:19:55.117315 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117288 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 16 18:19:55.117554 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117542 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="config-reloader" Apr 16 18:19:55.117609 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117556 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="config-reloader" Apr 16 18:19:55.117609 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117567 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy-thanos" Apr 16 18:19:55.117609 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117574 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy-thanos" Apr 16 18:19:55.117609 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117581 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="init-config-reloader" Apr 16 18:19:55.117609 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117587 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="init-config-reloader" Apr 16 18:19:55.117609 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117603 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="prometheus" Apr 16 18:19:55.117609 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117608 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="prometheus" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117613 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="thanos-sidecar" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117618 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="thanos-sidecar" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117657 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy-web" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117663 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy-web" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117670 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117675 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117719 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117726 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="prometheus" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117735 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="config-reloader" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117743 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy-thanos" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117750 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="kube-rbac-proxy-web" Apr 16 18:19:55.117890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.117756 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" containerName="thanos-sidecar" Apr 16 18:19:55.123444 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.123423 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.125822 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.125797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.125925 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.125849 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.125925 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.125892 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126026 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.125926 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126026 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.125947 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126026 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.125956 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Apr 16 18:19:55.126026 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.125956 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s96f7\"" Apr 16 18:19:55.126026 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.125994 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Apr 16 18:19:55.126026 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126004 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Apr 16 18:19:55.126026 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.125972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126031 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Apr 16 18:19:55.126357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126062 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126148 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxd22\" (UniqueName: \"kubernetes.io/projected/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-kube-api-access-nxd22\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126198 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Apr 16 18:19:55.126357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126204 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Apr 16 18:19:55.126357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126207 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Apr 16 18:19:55.126357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126203 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-config\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126317 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126205 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Apr 16 18:19:55.126357 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126358 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126830 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126397 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126830 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126421 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126830 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126446 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126830 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126830 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126558 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Apr 16 18:19:55.126830 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126573 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Apr 16 18:19:55.126830 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126582 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126830 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126683 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.126830 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.126720 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-jlpjbromi54k\"" Apr 16 18:19:55.129547 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.129529 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Apr 16 18:19:55.132736 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.132715 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Apr 16 18:19:55.136790 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.136766 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 16 18:19:55.228081 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228044 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228081 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228082 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228103 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228118 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228142 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228158 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228176 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228222 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nxd22\" (UniqueName: \"kubernetes.io/projected/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-kube-api-access-nxd22\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228265 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-config\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228290 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228349 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228328 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228798 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228365 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228798 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228389 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228798 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228417 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228798 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228444 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228798 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228477 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228798 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228512 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.228798 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.228579 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.229227 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.229190 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.229674 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.229335 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.231951 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.231542 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-web-config\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.231951 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.231567 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.231951 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.231801 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.231951 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.231816 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.231951 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.231830 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.231951 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.231834 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.231951 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.231892 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.232835 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.232811 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-config\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.232936 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.232833 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.233039 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.233021 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-config-out\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.235429 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.235358 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.236694 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.235554 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.236694 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.235569 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.236694 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.236137 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.240347 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.240323 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.241174 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.241157 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxd22\" (UniqueName: \"kubernetes.io/projected/22ae4dea-bd6e-4702-8500-e3dcb075fb4a-kube-api-access-nxd22\") pod \"prometheus-k8s-0\" (UID: \"22ae4dea-bd6e-4702-8500-e3dcb075fb4a\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.434890 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.434787 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:19:55.579742 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:55.579712 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 16 18:19:55.581146 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:19:55.581113 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ae4dea_bd6e_4702_8500_e3dcb075fb4a.slice/crio-adf3ffc5e0f408e9ebe3f7fc58096b23b5cd9418bae964573f92b9963c378085 WatchSource:0}: Error finding container adf3ffc5e0f408e9ebe3f7fc58096b23b5cd9418bae964573f92b9963c378085: Status 404 returned error can't find the container with id adf3ffc5e0f408e9ebe3f7fc58096b23b5cd9418bae964573f92b9963c378085 Apr 16 18:19:56.065242 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:56.065203 2577 generic.go:358] "Generic (PLEG): container finished" podID="22ae4dea-bd6e-4702-8500-e3dcb075fb4a" containerID="f36d2c131a7bdc3d92beceae52102749ef89f66aebad6c611af19f19298dc1d6" exitCode=0 Apr 16 18:19:56.065648 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:56.065299 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22ae4dea-bd6e-4702-8500-e3dcb075fb4a","Type":"ContainerDied","Data":"f36d2c131a7bdc3d92beceae52102749ef89f66aebad6c611af19f19298dc1d6"} Apr 16 18:19:56.065648 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:56.065335 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22ae4dea-bd6e-4702-8500-e3dcb075fb4a","Type":"ContainerStarted","Data":"adf3ffc5e0f408e9ebe3f7fc58096b23b5cd9418bae964573f92b9963c378085"} Apr 16 18:19:56.215043 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:56.215016 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d" path="/var/lib/kubelet/pods/436fc48f-9ecb-4b8d-a8a0-9a7fd7e5428d/volumes" Apr 16 18:19:57.071566 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:57.071523 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22ae4dea-bd6e-4702-8500-e3dcb075fb4a","Type":"ContainerStarted","Data":"9173a74db32b871e678dad0ba39c9ef4ef6de9af779e61419fc80403cb0a8d96"} Apr 16 18:19:57.071566 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:57.071568 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22ae4dea-bd6e-4702-8500-e3dcb075fb4a","Type":"ContainerStarted","Data":"70c8650fb2bea5b8d297ea0091e6180d5a405b09d69899478526e9dfa77bd36f"} Apr 16 18:19:57.072010 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:57.071582 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22ae4dea-bd6e-4702-8500-e3dcb075fb4a","Type":"ContainerStarted","Data":"bd7bb29419aaa1a65d02f57a37fbe6112b3f52e5a2e6368a7bf91de2f3b73ebb"} Apr 16 18:19:57.072010 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:57.071596 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22ae4dea-bd6e-4702-8500-e3dcb075fb4a","Type":"ContainerStarted","Data":"4266858f69ca3557ae917f87d55b7e977944354b9830090f99ed55eeffbaf2af"} Apr 16 18:19:57.072010 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:57.071607 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22ae4dea-bd6e-4702-8500-e3dcb075fb4a","Type":"ContainerStarted","Data":"b4a17bc43e054aa7aa0fc300f5ec3e175a466c6a203dce7ff6c0a717004119df"} Apr 16 18:19:57.072010 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:57.071619 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22ae4dea-bd6e-4702-8500-e3dcb075fb4a","Type":"ContainerStarted","Data":"041a55680ed120263e5b063c7f5214ec9512dcff879c62484c66d37ea52ac674"} Apr 16 18:19:57.098490 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:19:57.098426 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.098407165 podStartE2EDuration="2.098407165s" podCreationTimestamp="2026-04-16 18:19:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 18:19:57.097660165 +0000 UTC m=+261.491041413" watchObservedRunningTime="2026-04-16 18:19:57.098407165 +0000 UTC m=+261.491788423" Apr 16 18:20:00.435753 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:00.435715 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:20:23.018423 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.018387 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-jl946"] Apr 16 18:20:23.020531 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.020514 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jl946" Apr 16 18:20:23.022928 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.022904 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Apr 16 18:20:23.029237 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.029215 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jl946"] Apr 16 18:20:23.167419 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.167380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba22c77c-4d8b-4f6b-8230-ea8e843e78e5-kubelet-config\") pod \"global-pull-secret-syncer-jl946\" (UID: \"ba22c77c-4d8b-4f6b-8230-ea8e843e78e5\") " pod="kube-system/global-pull-secret-syncer-jl946" Apr 16 18:20:23.167419 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.167422 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba22c77c-4d8b-4f6b-8230-ea8e843e78e5-original-pull-secret\") pod \"global-pull-secret-syncer-jl946\" (UID: \"ba22c77c-4d8b-4f6b-8230-ea8e843e78e5\") " pod="kube-system/global-pull-secret-syncer-jl946" Apr 16 18:20:23.167673 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.167450 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba22c77c-4d8b-4f6b-8230-ea8e843e78e5-dbus\") pod \"global-pull-secret-syncer-jl946\" (UID: \"ba22c77c-4d8b-4f6b-8230-ea8e843e78e5\") " pod="kube-system/global-pull-secret-syncer-jl946" Apr 16 18:20:23.268235 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.268193 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba22c77c-4d8b-4f6b-8230-ea8e843e78e5-kubelet-config\") pod \"global-pull-secret-syncer-jl946\" (UID: \"ba22c77c-4d8b-4f6b-8230-ea8e843e78e5\") " pod="kube-system/global-pull-secret-syncer-jl946" Apr 16 18:20:23.268475 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.268330 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba22c77c-4d8b-4f6b-8230-ea8e843e78e5-original-pull-secret\") pod \"global-pull-secret-syncer-jl946\" (UID: \"ba22c77c-4d8b-4f6b-8230-ea8e843e78e5\") " pod="kube-system/global-pull-secret-syncer-jl946" Apr 16 18:20:23.268475 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.268389 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba22c77c-4d8b-4f6b-8230-ea8e843e78e5-dbus\") pod \"global-pull-secret-syncer-jl946\" (UID: \"ba22c77c-4d8b-4f6b-8230-ea8e843e78e5\") " pod="kube-system/global-pull-secret-syncer-jl946" Apr 16 18:20:23.268475 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.268334 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba22c77c-4d8b-4f6b-8230-ea8e843e78e5-kubelet-config\") pod \"global-pull-secret-syncer-jl946\" (UID: \"ba22c77c-4d8b-4f6b-8230-ea8e843e78e5\") " pod="kube-system/global-pull-secret-syncer-jl946" Apr 16 18:20:23.268668 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.268562 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba22c77c-4d8b-4f6b-8230-ea8e843e78e5-dbus\") pod \"global-pull-secret-syncer-jl946\" (UID: \"ba22c77c-4d8b-4f6b-8230-ea8e843e78e5\") " pod="kube-system/global-pull-secret-syncer-jl946" Apr 16 18:20:23.270769 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.270751 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba22c77c-4d8b-4f6b-8230-ea8e843e78e5-original-pull-secret\") pod \"global-pull-secret-syncer-jl946\" (UID: \"ba22c77c-4d8b-4f6b-8230-ea8e843e78e5\") " pod="kube-system/global-pull-secret-syncer-jl946" Apr 16 18:20:23.331204 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.331150 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jl946" Apr 16 18:20:23.460075 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:23.459986 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jl946"] Apr 16 18:20:23.462843 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:20:23.462815 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba22c77c_4d8b_4f6b_8230_ea8e843e78e5.slice/crio-5c74ddb322ca00ff634f57a73a9aff701263cf1bb48058ed3a1c772e6ea532ea WatchSource:0}: Error finding container 5c74ddb322ca00ff634f57a73a9aff701263cf1bb48058ed3a1c772e6ea532ea: Status 404 returned error can't find the container with id 5c74ddb322ca00ff634f57a73a9aff701263cf1bb48058ed3a1c772e6ea532ea Apr 16 18:20:24.154492 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:24.154453 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jl946" event={"ID":"ba22c77c-4d8b-4f6b-8230-ea8e843e78e5","Type":"ContainerStarted","Data":"5c74ddb322ca00ff634f57a73a9aff701263cf1bb48058ed3a1c772e6ea532ea"} Apr 16 18:20:28.168438 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:28.168404 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jl946" event={"ID":"ba22c77c-4d8b-4f6b-8230-ea8e843e78e5","Type":"ContainerStarted","Data":"a0fa4e7cba7fc93c490579fd7573f92520f87276bf18e5f2c9b68d344933b4d1"} Apr 16 18:20:28.182983 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:28.182844 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jl946" podStartSLOduration=2.51468053 podStartE2EDuration="6.182825348s" podCreationTimestamp="2026-04-16 18:20:22 +0000 UTC" firstStartedPulling="2026-04-16 18:20:23.464339193 +0000 UTC m=+287.857720428" lastFinishedPulling="2026-04-16 18:20:27.132484006 +0000 UTC m=+291.525865246" observedRunningTime="2026-04-16 18:20:28.182352272 +0000 UTC m=+292.575733529" watchObservedRunningTime="2026-04-16 18:20:28.182825348 +0000 UTC m=+292.576206605" Apr 16 18:20:36.085561 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:36.085528 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/2.log" Apr 16 18:20:36.085561 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:36.085558 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/2.log" Apr 16 18:20:36.098733 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:36.098709 2577 kubelet.go:1628] "Image garbage collection succeeded" Apr 16 18:20:55.435500 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:55.435450 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:20:55.451716 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:55.451689 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:20:56.267741 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:20:56.267712 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 16 18:22:31.159684 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.159648 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl"] Apr 16 18:22:31.161727 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.161711 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl" Apr 16 18:22:31.164022 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.163999 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Apr 16 18:22:31.164123 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.164106 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-h2vjq\"" Apr 16 18:22:31.164551 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.164535 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Apr 16 18:22:31.172491 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.172460 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl"] Apr 16 18:22:31.339368 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.339309 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvfvz\" (UniqueName: \"kubernetes.io/projected/d955dc83-769f-47ee-a045-df97566661ff-kube-api-access-fvfvz\") pod \"openshift-lws-operator-bfc7f696d-pfnxl\" (UID: \"d955dc83-769f-47ee-a045-df97566661ff\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl" Apr 16 18:22:31.339579 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.339434 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d955dc83-769f-47ee-a045-df97566661ff-tmp\") pod \"openshift-lws-operator-bfc7f696d-pfnxl\" (UID: \"d955dc83-769f-47ee-a045-df97566661ff\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl" Apr 16 18:22:31.440096 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.439984 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d955dc83-769f-47ee-a045-df97566661ff-tmp\") pod \"openshift-lws-operator-bfc7f696d-pfnxl\" (UID: \"d955dc83-769f-47ee-a045-df97566661ff\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl" Apr 16 18:22:31.440096 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.440073 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fvfvz\" (UniqueName: \"kubernetes.io/projected/d955dc83-769f-47ee-a045-df97566661ff-kube-api-access-fvfvz\") pod \"openshift-lws-operator-bfc7f696d-pfnxl\" (UID: \"d955dc83-769f-47ee-a045-df97566661ff\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl" Apr 16 18:22:31.440458 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.440431 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d955dc83-769f-47ee-a045-df97566661ff-tmp\") pod \"openshift-lws-operator-bfc7f696d-pfnxl\" (UID: \"d955dc83-769f-47ee-a045-df97566661ff\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl" Apr 16 18:22:31.448315 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.448282 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvfvz\" (UniqueName: \"kubernetes.io/projected/d955dc83-769f-47ee-a045-df97566661ff-kube-api-access-fvfvz\") pod \"openshift-lws-operator-bfc7f696d-pfnxl\" (UID: \"d955dc83-769f-47ee-a045-df97566661ff\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl" Apr 16 18:22:31.471191 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.471153 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl" Apr 16 18:22:31.595592 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.595566 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl"] Apr 16 18:22:31.598257 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:22:31.598225 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd955dc83_769f_47ee_a045_df97566661ff.slice/crio-c2f755c7424eed96f3172031297f9b33923c86c0bd15adec0d7ed406660c3f56 WatchSource:0}: Error finding container c2f755c7424eed96f3172031297f9b33923c86c0bd15adec0d7ed406660c3f56: Status 404 returned error can't find the container with id c2f755c7424eed96f3172031297f9b33923c86c0bd15adec0d7ed406660c3f56 Apr 16 18:22:31.599550 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:31.599533 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 18:22:32.529320 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:32.529276 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl" event={"ID":"d955dc83-769f-47ee-a045-df97566661ff","Type":"ContainerStarted","Data":"c2f755c7424eed96f3172031297f9b33923c86c0bd15adec0d7ed406660c3f56"} Apr 16 18:22:34.539640 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:34.539581 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl" event={"ID":"d955dc83-769f-47ee-a045-df97566661ff","Type":"ContainerStarted","Data":"b744d84ed0c35cfd540ccf9f9ec39554ca1c9568acb109219f1707a222d0c9f4"} Apr 16 18:22:34.555827 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:22:34.555769 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-pfnxl" podStartSLOduration=1.273543098 podStartE2EDuration="3.555752215s" podCreationTimestamp="2026-04-16 18:22:31 +0000 UTC" firstStartedPulling="2026-04-16 18:22:31.599683399 +0000 UTC m=+415.993064635" lastFinishedPulling="2026-04-16 18:22:33.881892506 +0000 UTC m=+418.275273752" observedRunningTime="2026-04-16 18:22:34.553936211 +0000 UTC m=+418.947317468" watchObservedRunningTime="2026-04-16 18:22:34.555752215 +0000 UTC m=+418.949133486" Apr 16 18:23:13.064336 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.064299 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg"] Apr 16 18:23:13.067684 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.067665 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.070779 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.070750 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Apr 16 18:23:13.070909 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.070754 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hzx2r\"" Apr 16 18:23:13.070909 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.070797 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Apr 16 18:23:13.070909 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.070765 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Apr 16 18:23:13.078589 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.078566 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg"] Apr 16 18:23:13.185251 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.185211 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c6b709aa-c224-489e-aa6c-927216f7c61d-manager-config\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.185450 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.185301 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2drw\" (UniqueName: \"kubernetes.io/projected/c6b709aa-c224-489e-aa6c-927216f7c61d-kube-api-access-z2drw\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.185450 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.185332 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6b709aa-c224-489e-aa6c-927216f7c61d-cert\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.185450 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.185347 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6b709aa-c224-489e-aa6c-927216f7c61d-metrics-cert\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.286023 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.285984 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c6b709aa-c224-489e-aa6c-927216f7c61d-manager-config\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.286219 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.286071 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z2drw\" (UniqueName: \"kubernetes.io/projected/c6b709aa-c224-489e-aa6c-927216f7c61d-kube-api-access-z2drw\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.286219 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.286107 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6b709aa-c224-489e-aa6c-927216f7c61d-cert\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.286219 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.286129 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6b709aa-c224-489e-aa6c-927216f7c61d-metrics-cert\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.286780 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.286755 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c6b709aa-c224-489e-aa6c-927216f7c61d-manager-config\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.288741 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.288720 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6b709aa-c224-489e-aa6c-927216f7c61d-metrics-cert\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.288869 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.288851 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6b709aa-c224-489e-aa6c-927216f7c61d-cert\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.309995 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.309967 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2drw\" (UniqueName: \"kubernetes.io/projected/c6b709aa-c224-489e-aa6c-927216f7c61d-kube-api-access-z2drw\") pod \"lws-controller-manager-5d577d5585-cdvbg\" (UID: \"c6b709aa-c224-489e-aa6c-927216f7c61d\") " pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.377394 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.377358 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:13.504584 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.504550 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg"] Apr 16 18:23:13.507578 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:23:13.507542 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b709aa_c224_489e_aa6c_927216f7c61d.slice/crio-011709b9efe3823ff70f67248db7be01c0fd8353502dfa12892678d893271132 WatchSource:0}: Error finding container 011709b9efe3823ff70f67248db7be01c0fd8353502dfa12892678d893271132: Status 404 returned error can't find the container with id 011709b9efe3823ff70f67248db7be01c0fd8353502dfa12892678d893271132 Apr 16 18:23:13.658764 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:13.658668 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" event={"ID":"c6b709aa-c224-489e-aa6c-927216f7c61d","Type":"ContainerStarted","Data":"011709b9efe3823ff70f67248db7be01c0fd8353502dfa12892678d893271132"} Apr 16 18:23:15.666918 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:15.666878 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" event={"ID":"c6b709aa-c224-489e-aa6c-927216f7c61d","Type":"ContainerStarted","Data":"65d706f2e60830eea90a114e68be67e36aec67dc6fb8f34a32442ac5cb27d3b1"} Apr 16 18:23:15.667355 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:15.666941 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:15.683381 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:15.683328 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" podStartSLOduration=1.061784324 podStartE2EDuration="2.683314313s" podCreationTimestamp="2026-04-16 18:23:13 +0000 UTC" firstStartedPulling="2026-04-16 18:23:13.509438987 +0000 UTC m=+457.902820225" lastFinishedPulling="2026-04-16 18:23:15.130968974 +0000 UTC m=+459.524350214" observedRunningTime="2026-04-16 18:23:15.683309941 +0000 UTC m=+460.076691197" watchObservedRunningTime="2026-04-16 18:23:15.683314313 +0000 UTC m=+460.076695644" Apr 16 18:23:26.672377 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:26.672344 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5d577d5585-cdvbg" Apr 16 18:23:41.315814 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.315780 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg"] Apr 16 18:23:41.318025 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.318007 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" Apr 16 18:23:41.320198 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.320173 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Apr 16 18:23:41.320317 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.320298 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Apr 16 18:23:41.320458 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.320443 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qnh87\"" Apr 16 18:23:41.330007 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.329984 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg"] Apr 16 18:23:41.427737 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.427693 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8g54\" (UniqueName: \"kubernetes.io/projected/b15effd1-2a50-46ee-8de5-1d8278e79fb5-kube-api-access-d8g54\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-dtskg\" (UID: \"b15effd1-2a50-46ee-8de5-1d8278e79fb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" Apr 16 18:23:41.427912 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.427760 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b15effd1-2a50-46ee-8de5-1d8278e79fb5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-dtskg\" (UID: \"b15effd1-2a50-46ee-8de5-1d8278e79fb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" Apr 16 18:23:41.528348 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.528311 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d8g54\" (UniqueName: \"kubernetes.io/projected/b15effd1-2a50-46ee-8de5-1d8278e79fb5-kube-api-access-d8g54\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-dtskg\" (UID: \"b15effd1-2a50-46ee-8de5-1d8278e79fb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" Apr 16 18:23:41.528513 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.528371 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b15effd1-2a50-46ee-8de5-1d8278e79fb5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-dtskg\" (UID: \"b15effd1-2a50-46ee-8de5-1d8278e79fb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" Apr 16 18:23:41.528733 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.528718 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b15effd1-2a50-46ee-8de5-1d8278e79fb5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-dtskg\" (UID: \"b15effd1-2a50-46ee-8de5-1d8278e79fb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" Apr 16 18:23:41.540603 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.540572 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8g54\" (UniqueName: \"kubernetes.io/projected/b15effd1-2a50-46ee-8de5-1d8278e79fb5-kube-api-access-d8g54\") pod \"kuadrant-operator-controller-manager-6ddf9554fc-dtskg\" (UID: \"b15effd1-2a50-46ee-8de5-1d8278e79fb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" Apr 16 18:23:41.628208 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.628121 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" Apr 16 18:23:41.771885 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:41.771853 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg"] Apr 16 18:23:41.774947 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:23:41.774916 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15effd1_2a50_46ee_8de5_1d8278e79fb5.slice/crio-52bbd3987d40db67574977c42f5d786f4cca1c0cde079c8ce3ffcf7b8551f193 WatchSource:0}: Error finding container 52bbd3987d40db67574977c42f5d786f4cca1c0cde079c8ce3ffcf7b8551f193: Status 404 returned error can't find the container with id 52bbd3987d40db67574977c42f5d786f4cca1c0cde079c8ce3ffcf7b8551f193 Apr 16 18:23:42.748672 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:42.748600 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" event={"ID":"b15effd1-2a50-46ee-8de5-1d8278e79fb5","Type":"ContainerStarted","Data":"52bbd3987d40db67574977c42f5d786f4cca1c0cde079c8ce3ffcf7b8551f193"} Apr 16 18:23:45.761821 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:45.761728 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" event={"ID":"b15effd1-2a50-46ee-8de5-1d8278e79fb5","Type":"ContainerStarted","Data":"20ddd4519329ebe594cafaeaa1d9fb83efa2fe2f5af1908d2ebd27014e16f14d"} Apr 16 18:23:45.762263 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:45.761834 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" Apr 16 18:23:45.784797 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:45.784736 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" podStartSLOduration=1.096208549 podStartE2EDuration="4.784719663s" podCreationTimestamp="2026-04-16 18:23:41 +0000 UTC" firstStartedPulling="2026-04-16 18:23:41.777945276 +0000 UTC m=+486.171326511" lastFinishedPulling="2026-04-16 18:23:45.466456377 +0000 UTC m=+489.859837625" observedRunningTime="2026-04-16 18:23:45.783783536 +0000 UTC m=+490.177164830" watchObservedRunningTime="2026-04-16 18:23:45.784719663 +0000 UTC m=+490.178100919" Apr 16 18:23:56.768490 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:23:56.768460 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-dtskg" Apr 16 18:25:36.110127 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:25:36.110098 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/2.log" Apr 16 18:25:36.112390 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:25:36.112369 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/2.log" Apr 16 18:30:36.130783 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:30:36.130705 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/2.log" Apr 16 18:30:36.135259 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:30:36.135235 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/2.log" Apr 16 18:34:01.351868 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.351832 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5b6zt/must-gather-bmq5f"] Apr 16 18:34:01.353887 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.353871 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5b6zt/must-gather-bmq5f" Apr 16 18:34:01.356296 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.356275 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5b6zt\"/\"kube-root-ca.crt\"" Apr 16 18:34:01.356428 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.356323 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5b6zt\"/\"openshift-service-ca.crt\"" Apr 16 18:34:01.356428 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.356284 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5b6zt\"/\"default-dockercfg-nlsgc\"" Apr 16 18:34:01.365227 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.365204 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5b6zt/must-gather-bmq5f"] Apr 16 18:34:01.472920 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.472872 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7fedafcf-fc75-4e61-805c-90ed6b3cdef9-must-gather-output\") pod \"must-gather-bmq5f\" (UID: \"7fedafcf-fc75-4e61-805c-90ed6b3cdef9\") " pod="openshift-must-gather-5b6zt/must-gather-bmq5f" Apr 16 18:34:01.472920 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.472921 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z4bh\" (UniqueName: \"kubernetes.io/projected/7fedafcf-fc75-4e61-805c-90ed6b3cdef9-kube-api-access-9z4bh\") pod \"must-gather-bmq5f\" (UID: \"7fedafcf-fc75-4e61-805c-90ed6b3cdef9\") " pod="openshift-must-gather-5b6zt/must-gather-bmq5f" Apr 16 18:34:01.574202 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.574162 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9z4bh\" (UniqueName: \"kubernetes.io/projected/7fedafcf-fc75-4e61-805c-90ed6b3cdef9-kube-api-access-9z4bh\") pod \"must-gather-bmq5f\" (UID: \"7fedafcf-fc75-4e61-805c-90ed6b3cdef9\") " pod="openshift-must-gather-5b6zt/must-gather-bmq5f" Apr 16 18:34:01.574352 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.574270 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7fedafcf-fc75-4e61-805c-90ed6b3cdef9-must-gather-output\") pod \"must-gather-bmq5f\" (UID: \"7fedafcf-fc75-4e61-805c-90ed6b3cdef9\") " pod="openshift-must-gather-5b6zt/must-gather-bmq5f" Apr 16 18:34:01.574556 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.574541 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7fedafcf-fc75-4e61-805c-90ed6b3cdef9-must-gather-output\") pod \"must-gather-bmq5f\" (UID: \"7fedafcf-fc75-4e61-805c-90ed6b3cdef9\") " pod="openshift-must-gather-5b6zt/must-gather-bmq5f" Apr 16 18:34:01.583338 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.583303 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z4bh\" (UniqueName: \"kubernetes.io/projected/7fedafcf-fc75-4e61-805c-90ed6b3cdef9-kube-api-access-9z4bh\") pod \"must-gather-bmq5f\" (UID: \"7fedafcf-fc75-4e61-805c-90ed6b3cdef9\") " pod="openshift-must-gather-5b6zt/must-gather-bmq5f" Apr 16 18:34:01.663288 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.663197 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5b6zt/must-gather-bmq5f" Apr 16 18:34:01.795369 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.795334 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5b6zt/must-gather-bmq5f"] Apr 16 18:34:01.798725 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:34:01.798692 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fedafcf_fc75_4e61_805c_90ed6b3cdef9.slice/crio-3c2e617af14e6c9520e2d673c82e8d0c31a95dbfd93fa5ca560257e5f4ad5fc1 WatchSource:0}: Error finding container 3c2e617af14e6c9520e2d673c82e8d0c31a95dbfd93fa5ca560257e5f4ad5fc1: Status 404 returned error can't find the container with id 3c2e617af14e6c9520e2d673c82e8d0c31a95dbfd93fa5ca560257e5f4ad5fc1 Apr 16 18:34:01.800420 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:01.800405 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 18:34:02.769595 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:02.769558 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5b6zt/must-gather-bmq5f" event={"ID":"7fedafcf-fc75-4e61-805c-90ed6b3cdef9","Type":"ContainerStarted","Data":"13fad4aff9a2c14de0a328bb61fa8fdc012b23bfac4b85f8f9eef66fba700a38"} Apr 16 18:34:02.769595 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:02.769594 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5b6zt/must-gather-bmq5f" event={"ID":"7fedafcf-fc75-4e61-805c-90ed6b3cdef9","Type":"ContainerStarted","Data":"3c2e617af14e6c9520e2d673c82e8d0c31a95dbfd93fa5ca560257e5f4ad5fc1"} Apr 16 18:34:03.776847 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:03.776797 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5b6zt/must-gather-bmq5f" event={"ID":"7fedafcf-fc75-4e61-805c-90ed6b3cdef9","Type":"ContainerStarted","Data":"30e50b842165b4a914087f5b395d56212647e66f4f85a1c90ad0c5dcaa11fdf0"} Apr 16 18:34:03.794651 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:03.794562 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-5b6zt/must-gather-bmq5f" podStartSLOduration=2.061958818 podStartE2EDuration="2.794541361s" podCreationTimestamp="2026-04-16 18:34:01 +0000 UTC" firstStartedPulling="2026-04-16 18:34:01.800539164 +0000 UTC m=+1106.193920399" lastFinishedPulling="2026-04-16 18:34:02.533121707 +0000 UTC m=+1106.926502942" observedRunningTime="2026-04-16 18:34:03.794004397 +0000 UTC m=+1108.187385657" watchObservedRunningTime="2026-04-16 18:34:03.794541361 +0000 UTC m=+1108.187922619" Apr 16 18:34:04.094849 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:04.094770 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jl946_ba22c77c-4d8b-4f6b-8230-ea8e843e78e5/global-pull-secret-syncer/0.log" Apr 16 18:34:04.301901 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:04.301867 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vhpvh_87f2496f-9291-4513-812e-23f4132ec75c/konnectivity-agent/0.log" Apr 16 18:34:04.387244 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:04.387169 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-22.ec2.internal_9651c74f7f561de9ee59b0475859f99e/haproxy/0.log" Apr 16 18:34:08.455473 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:08.455393 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_462b34a0-4db7-486a-9db0-bc0614828026/alertmanager/0.log" Apr 16 18:34:08.487005 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:08.486972 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_462b34a0-4db7-486a-9db0-bc0614828026/config-reloader/0.log" Apr 16 18:34:08.514212 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:08.514172 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_462b34a0-4db7-486a-9db0-bc0614828026/kube-rbac-proxy-web/0.log" Apr 16 18:34:08.547603 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:08.547576 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_462b34a0-4db7-486a-9db0-bc0614828026/kube-rbac-proxy/0.log" Apr 16 18:34:08.575163 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:08.575136 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_462b34a0-4db7-486a-9db0-bc0614828026/kube-rbac-proxy-metric/0.log" Apr 16 18:34:08.610520 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:08.610494 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_462b34a0-4db7-486a-9db0-bc0614828026/prom-label-proxy/0.log" Apr 16 18:34:08.642837 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:08.642765 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_462b34a0-4db7-486a-9db0-bc0614828026/init-config-reloader/0.log" Apr 16 18:34:08.816074 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:08.816037 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-579b89d77b-6zfgc_11ffe715-7a8a-4c09-94b6-e76e7ed8b6b4/metrics-server/0.log" Apr 16 18:34:08.888144 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:08.888101 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4b4th_3f917f86-7a73-4822-9bcf-6f83983701cd/node-exporter/0.log" Apr 16 18:34:08.919463 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:08.919437 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4b4th_3f917f86-7a73-4822-9bcf-6f83983701cd/kube-rbac-proxy/0.log" Apr 16 18:34:08.954726 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:08.954698 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4b4th_3f917f86-7a73-4822-9bcf-6f83983701cd/init-textfile/0.log" Apr 16 18:34:09.290181 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:09.290150 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22ae4dea-bd6e-4702-8500-e3dcb075fb4a/prometheus/0.log" Apr 16 18:34:09.314985 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:09.314955 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22ae4dea-bd6e-4702-8500-e3dcb075fb4a/config-reloader/0.log" Apr 16 18:34:09.350249 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:09.350183 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22ae4dea-bd6e-4702-8500-e3dcb075fb4a/thanos-sidecar/0.log" Apr 16 18:34:09.381182 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:09.381116 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22ae4dea-bd6e-4702-8500-e3dcb075fb4a/kube-rbac-proxy-web/0.log" Apr 16 18:34:09.410948 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:09.410919 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22ae4dea-bd6e-4702-8500-e3dcb075fb4a/kube-rbac-proxy/0.log" Apr 16 18:34:09.439433 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:09.439401 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22ae4dea-bd6e-4702-8500-e3dcb075fb4a/kube-rbac-proxy-thanos/0.log" Apr 16 18:34:09.469964 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:09.469926 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22ae4dea-bd6e-4702-8500-e3dcb075fb4a/init-config-reloader/0.log" Apr 16 18:34:09.608871 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:09.608843 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6566964757-bvqbw_1af84565-9bdb-4940-9991-d03274125619/telemeter-client/0.log" Apr 16 18:34:09.641426 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:09.641397 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6566964757-bvqbw_1af84565-9bdb-4940-9991-d03274125619/reload/0.log" Apr 16 18:34:09.672468 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:09.672430 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6566964757-bvqbw_1af84565-9bdb-4940-9991-d03274125619/kube-rbac-proxy/0.log" Apr 16 18:34:11.947717 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:11.947596 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/2.log" Apr 16 18:34:11.952823 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:11.952798 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-l9q5d_14efee5f-03ef-4047-86f8-ede90b8b5238/console-operator/3.log" Apr 16 18:34:13.179707 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.179672 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2"] Apr 16 18:34:13.186683 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.186656 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.194003 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.193975 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2"] Apr 16 18:34:13.205728 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.205690 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xjqk\" (UniqueName: \"kubernetes.io/projected/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-kube-api-access-8xjqk\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.206038 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.206015 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-sys\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.206200 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.206186 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-proc\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.206331 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.206317 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-lib-modules\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.206452 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.206438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-podres\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.308046 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.308004 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-sys\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.308234 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.308090 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-proc\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.308234 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.308160 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-lib-modules\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.308234 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.308201 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-podres\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.308406 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.308256 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8xjqk\" (UniqueName: \"kubernetes.io/projected/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-kube-api-access-8xjqk\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.308850 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.308823 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-proc\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.308969 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.308811 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-sys\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.308969 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.308911 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-podres\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.308969 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.308911 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-lib-modules\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.317770 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.317744 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xjqk\" (UniqueName: \"kubernetes.io/projected/3a7b4768-a8ca-459e-a53c-8535b07a5a9b-kube-api-access-8xjqk\") pod \"perf-node-gather-daemonset-xkzv2\" (UID: \"3a7b4768-a8ca-459e-a53c-8535b07a5a9b\") " pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.500690 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.500563 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.646847 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.646771 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2"] Apr 16 18:34:13.657464 ip-10-0-138-22 kubenswrapper[2577]: W0416 18:34:13.657422 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3a7b4768_a8ca_459e_a53c_8535b07a5a9b.slice/crio-b98197bbcdf73cab54462d064d356cbdd2f8ed16a78bbef02d7fb4583beda54a WatchSource:0}: Error finding container b98197bbcdf73cab54462d064d356cbdd2f8ed16a78bbef02d7fb4583beda54a: Status 404 returned error can't find the container with id b98197bbcdf73cab54462d064d356cbdd2f8ed16a78bbef02d7fb4583beda54a Apr 16 18:34:13.786904 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.786881 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7mtp7_e7a02fec-362b-4be1-bf77-0fbf2a0a986e/dns/0.log" Apr 16 18:34:13.814651 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.814596 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7mtp7_e7a02fec-362b-4be1-bf77-0fbf2a0a986e/kube-rbac-proxy/0.log" Apr 16 18:34:13.817718 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.817690 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" event={"ID":"3a7b4768-a8ca-459e-a53c-8535b07a5a9b","Type":"ContainerStarted","Data":"81593035a4d761754284846eaec5719a25b8912d935e1e71e273ae37e3a2cc3b"} Apr 16 18:34:13.817865 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.817727 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" event={"ID":"3a7b4768-a8ca-459e-a53c-8535b07a5a9b","Type":"ContainerStarted","Data":"b98197bbcdf73cab54462d064d356cbdd2f8ed16a78bbef02d7fb4583beda54a"} Apr 16 18:34:13.817865 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.817761 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:13.836597 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:13.836544 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" podStartSLOduration=0.836528297 podStartE2EDuration="836.528297ms" podCreationTimestamp="2026-04-16 18:34:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 18:34:13.835086332 +0000 UTC m=+1118.228467586" watchObservedRunningTime="2026-04-16 18:34:13.836528297 +0000 UTC m=+1118.229909590" Apr 16 18:34:14.019331 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:14.019301 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kdfqc_59928e2c-472c-4289-b77d-213f2811bb82/dns-node-resolver/0.log" Apr 16 18:34:14.589949 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:14.589914 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wh4nz_109afb78-2acc-4972-9604-32fb10a7ae6b/node-ca/0.log" Apr 16 18:34:16.023546 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:16.023516 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fqnj7_1f3ce71d-67a7-431e-9a65-37dd55851b2c/serve-healthcheck-canary/0.log" Apr 16 18:34:16.499775 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:16.499689 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4rkbt_203100f0-5c6d-4372-a64b-f2823d3d7d55/kube-rbac-proxy/0.log" Apr 16 18:34:16.528278 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:16.528247 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4rkbt_203100f0-5c6d-4372-a64b-f2823d3d7d55/exporter/0.log" Apr 16 18:34:16.555127 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:16.555096 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4rkbt_203100f0-5c6d-4372-a64b-f2823d3d7d55/extractor/0.log" Apr 16 18:34:18.986442 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:18.986413 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5d577d5585-cdvbg_c6b709aa-c224-489e-aa6c-927216f7c61d/manager/0.log" Apr 16 18:34:19.015771 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:19.015735 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-pfnxl_d955dc83-769f-47ee-a045-df97566661ff/openshift-lws-operator/0.log" Apr 16 18:34:19.832038 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:19.832010 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5b6zt/perf-node-gather-daemonset-xkzv2" Apr 16 18:34:24.683114 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:24.682942 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-69k7d_d6e9b61c-b5b2-400a-93ad-c16e7c4989b8/kube-multus/0.log" Apr 16 18:34:24.768053 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:24.768026 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-59lzl_9db69d07-9153-4912-af0e-5a73aa0f3f33/kube-multus-additional-cni-plugins/0.log" Apr 16 18:34:24.796062 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:24.796033 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-59lzl_9db69d07-9153-4912-af0e-5a73aa0f3f33/egress-router-binary-copy/0.log" Apr 16 18:34:24.822040 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:24.822014 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-59lzl_9db69d07-9153-4912-af0e-5a73aa0f3f33/cni-plugins/0.log" Apr 16 18:34:24.848058 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:24.848034 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-59lzl_9db69d07-9153-4912-af0e-5a73aa0f3f33/bond-cni-plugin/0.log" Apr 16 18:34:24.873752 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:24.873724 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-59lzl_9db69d07-9153-4912-af0e-5a73aa0f3f33/routeoverride-cni/0.log" Apr 16 18:34:24.898774 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:24.898737 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-59lzl_9db69d07-9153-4912-af0e-5a73aa0f3f33/whereabouts-cni-bincopy/0.log" Apr 16 18:34:24.928499 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:24.928468 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-59lzl_9db69d07-9153-4912-af0e-5a73aa0f3f33/whereabouts-cni/0.log" Apr 16 18:34:25.474482 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:25.474407 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lp8x6_5609e075-d6f5-4642-a711-20f608288876/network-metrics-daemon/0.log" Apr 16 18:34:25.499250 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:25.499223 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lp8x6_5609e075-d6f5-4642-a711-20f608288876/kube-rbac-proxy/0.log" Apr 16 18:34:27.074028 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:27.073979 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgfh9_e4a39fc7-9f68-446d-8cf6-04b77ee87df2/ovn-controller/0.log" Apr 16 18:34:27.109301 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:27.109272 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgfh9_e4a39fc7-9f68-446d-8cf6-04b77ee87df2/ovn-acl-logging/0.log" Apr 16 18:34:27.134352 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:27.134325 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgfh9_e4a39fc7-9f68-446d-8cf6-04b77ee87df2/kube-rbac-proxy-node/0.log" Apr 16 18:34:27.167392 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:27.167365 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgfh9_e4a39fc7-9f68-446d-8cf6-04b77ee87df2/kube-rbac-proxy-ovn-metrics/0.log" Apr 16 18:34:27.192229 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:27.192183 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgfh9_e4a39fc7-9f68-446d-8cf6-04b77ee87df2/northd/0.log" Apr 16 18:34:27.218344 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:27.218313 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgfh9_e4a39fc7-9f68-446d-8cf6-04b77ee87df2/nbdb/0.log" Apr 16 18:34:27.247843 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:27.247818 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgfh9_e4a39fc7-9f68-446d-8cf6-04b77ee87df2/sbdb/0.log" Apr 16 18:34:27.366478 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:27.366410 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgfh9_e4a39fc7-9f68-446d-8cf6-04b77ee87df2/ovnkube-controller/0.log" Apr 16 18:34:28.523103 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:28.523074 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-zf6t5_9792d3fc-f6a7-4d56-a906-b650dea53fb2/network-check-target-container/0.log" Apr 16 18:34:29.623240 ip-10-0-138-22 kubenswrapper[2577]: I0416 18:34:29.623210 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-25ptb_261d6b5b-fcc0-49a2-a408-af233d788aa3/iptables-alerter/0.log"