Jun 02 19:01:07.631471 ip-10-0-138-102 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 02 19:01:07.631485 ip-10-0-138-102 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 02 19:01:07.631494 ip-10-0-138-102 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 02 19:01:07.631825 ip-10-0-138-102 systemd[1]: Failed to start Kubernetes Kubelet. Jun 02 19:01:17.726756 ip-10-0-138-102 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 02 19:01:17.726775 ip-10-0-138-102 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a51bf520cd354f708840f89ecd98055a -- Jun 02 19:03:57.075483 ip-10-0-138-102 systemd[1]: Starting Kubernetes Kubelet... Jun 02 19:03:57.456420 ip-10-0-138-102 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 19:03:57.456420 ip-10-0-138-102 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 19:03:57.456420 ip-10-0-138-102 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 19:03:57.456420 ip-10-0-138-102 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 19:03:57.456420 ip-10-0-138-102 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 19:03:57.457758 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.457687 2574 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 19:03:57.461104 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461085 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:03:57.461104 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461101 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:03:57.461104 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461106 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:03:57.461104 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461110 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461116 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461122 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461128 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461132 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461136 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461141 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461145 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461149 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461153 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461157 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461162 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461166 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461170 2574 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461174 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461179 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461195 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461200 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461203 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:03:57.461357 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461207 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461227 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461232 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461236 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461240 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461244 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461250 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461254 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461259 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461264 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461268 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461272 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461276 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461281 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461285 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461291 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461295 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461300 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461304 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461309 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:03:57.462140 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461313 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461316 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461320 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461325 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461329 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461333 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461337 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461341 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461346 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461351 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461355 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461359 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461362 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461367 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461371 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461375 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461379 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461383 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461387 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461392 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:03:57.463010 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461396 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461400 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461404 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461409 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461413 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461418 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461422 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461426 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461430 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461434 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461439 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461443 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461447 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461454 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461458 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461462 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461466 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461471 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461475 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:03:57.463881 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461481 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461485 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461489 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461494 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.461500 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462092 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462100 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462105 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462110 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462115 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462119 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462126 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462131 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462135 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462139 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462143 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462147 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462151 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:03:57.464435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462156 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462161 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462165 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462169 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462174 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462178 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462182 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462186 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462191 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462195 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462199 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462203 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462207 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462229 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462234 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462238 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462243 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462247 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462252 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462256 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:03:57.464937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462260 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462264 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462268 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462272 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462276 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462281 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462285 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462289 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462293 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462297 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462302 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462306 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462311 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462315 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462319 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462323 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462327 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462331 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462335 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462339 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:03:57.465641 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462343 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462349 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462353 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462357 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462361 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462365 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462369 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462373 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462377 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462381 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462385 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462389 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462393 2574 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462397 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462401 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462405 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462410 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462416 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462422 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462426 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:03:57.466163 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462430 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462434 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462443 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462448 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462452 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462456 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462460 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462465 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462469 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462472 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462477 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462480 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.462485 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462582 2574 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462594 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462604 2574 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462611 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462618 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462623 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462635 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462641 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 19:03:57.466755 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462646 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462651 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462657 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462662 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462667 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462673 2574 flags.go:64] FLAG: --cgroup-root="" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462677 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462682 2574 flags.go:64] FLAG: --client-ca-file="" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462686 2574 flags.go:64] FLAG: --cloud-config="" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462691 2574 flags.go:64] FLAG: --cloud-provider="external" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462696 2574 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462702 2574 flags.go:64] FLAG: --cluster-domain="" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462707 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462714 2574 flags.go:64] FLAG: --config-dir="" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462719 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462724 2574 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462730 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462735 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462741 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462746 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462751 2574 flags.go:64] FLAG: --contention-profiling="false" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462755 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462760 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462765 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462770 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 19:03:57.467346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462776 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462783 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462788 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462792 2574 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462797 2574 flags.go:64] FLAG: --enable-server="true" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462802 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462810 2574 flags.go:64] FLAG: --event-burst="100" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462815 2574 flags.go:64] FLAG: --event-qps="50" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462820 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462826 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462831 2574 flags.go:64] FLAG: --eviction-hard="" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462836 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462841 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462846 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462850 2574 flags.go:64] FLAG: --eviction-soft="" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462855 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462860 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462865 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462870 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462874 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462880 2574 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462885 2574 flags.go:64] FLAG: --feature-gates="" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462892 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462897 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462903 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 19:03:57.467968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462908 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462913 2574 flags.go:64] FLAG: --healthz-port="10248" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462919 2574 flags.go:64] FLAG: --help="false" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462923 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-138-102.ec2.internal" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462928 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462933 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462938 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462944 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462950 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462955 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462960 2574 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462964 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462969 2574 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462974 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462980 2574 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462984 2574 flags.go:64] FLAG: --kube-reserved="" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462989 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462994 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.462999 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463004 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463008 2574 flags.go:64] FLAG: --lock-file="" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463013 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463018 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463023 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 19:03:57.468630 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463031 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463036 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463040 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463048 2574 flags.go:64] FLAG: --logging-format="text" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463053 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463059 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463064 2574 flags.go:64] FLAG: --manifest-url="" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463068 2574 flags.go:64] FLAG: --manifest-url-header="" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463075 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463080 2574 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463087 2574 flags.go:64] FLAG: --max-pods="110" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463092 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463096 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463101 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463106 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463111 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463116 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463120 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463133 2574 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463138 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463142 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463147 2574 flags.go:64] FLAG: --pod-cidr="" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463152 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 19:03:57.469269 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463160 2574 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463165 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463170 2574 flags.go:64] FLAG: --pods-per-core="0" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463175 2574 flags.go:64] FLAG: --port="10250" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463179 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463184 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07f0fdf7f01e0342f" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463190 2574 flags.go:64] FLAG: --qos-reserved="" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463195 2574 flags.go:64] FLAG: --read-only-port="10255" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463200 2574 flags.go:64] FLAG: --register-node="true" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463205 2574 flags.go:64] FLAG: --register-schedulable="true" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463210 2574 flags.go:64] FLAG: --register-with-taints="" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463232 2574 flags.go:64] FLAG: --registry-burst="10" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463237 2574 flags.go:64] FLAG: --registry-qps="5" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463245 2574 flags.go:64] FLAG: --reserved-cpus="" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463250 2574 flags.go:64] FLAG: --reserved-memory="" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463256 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463262 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463266 2574 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463271 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463276 2574 flags.go:64] FLAG: --runonce="false" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463281 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463286 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463291 2574 flags.go:64] FLAG: --seccomp-default="false" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463296 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463301 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463306 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 19:03:57.469828 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463311 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463316 2574 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463321 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463326 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463331 2574 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463336 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463340 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463345 2574 flags.go:64] FLAG: --system-cgroups="" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463349 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463358 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463363 2574 flags.go:64] FLAG: --tls-cert-file="" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463367 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463405 2574 flags.go:64] FLAG: --tls-min-version="" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463411 2574 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463416 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463421 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463427 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463432 2574 flags.go:64] FLAG: --v="2" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463439 2574 flags.go:64] FLAG: --version="false" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463452 2574 flags.go:64] FLAG: --vmodule="" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463459 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.463464 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463608 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463615 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:03:57.470510 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463620 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463625 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463629 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463634 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463639 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463643 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463647 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463651 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463656 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463660 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463664 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463669 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463673 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463678 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463682 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463686 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463691 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463695 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463699 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:03:57.471084 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463703 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463707 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463711 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463715 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463719 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463724 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463729 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463733 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463742 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463746 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463750 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463754 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463758 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463763 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463767 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463771 2574 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463775 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463779 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463783 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463788 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463792 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:03:57.471591 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463796 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463802 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463808 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463814 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463819 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463825 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463832 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463837 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463842 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463846 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463850 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463855 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463859 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463863 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463867 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463871 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463875 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463879 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463884 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:03:57.472108 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463892 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463897 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463901 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463905 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463910 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463914 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463918 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463923 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463927 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463931 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463936 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463940 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463944 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463948 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463953 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463957 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463961 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463965 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463970 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:03:57.472593 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463975 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463979 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463983 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463987 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463991 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.463995 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.464589 2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.470539 2574 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.470554 2574 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470599 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470606 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470611 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470616 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470620 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470622 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470625 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:03:57.473049 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470627 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470630 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470633 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470635 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470638 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470640 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470643 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470645 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470648 2574 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470650 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470653 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470656 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470658 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470661 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470664 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470667 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470670 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470672 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470675 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470678 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:03:57.473484 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470682 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470686 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470691 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470694 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470697 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470700 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470702 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470705 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470707 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470710 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470713 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470715 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470718 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470720 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470723 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470725 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470728 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470730 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470733 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:03:57.473978 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470736 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470740 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470743 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470746 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470748 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470751 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470753 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470757 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470761 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470766 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470770 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470772 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470775 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470778 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470780 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470783 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470786 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470789 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470792 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:03:57.474465 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470794 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470797 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470799 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470803 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470807 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470810 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470813 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470816 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470819 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470822 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470824 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470827 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470831 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470835 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470840 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470844 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470847 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470849 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470852 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470854 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:03:57.474938 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470857 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.470862 2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470960 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470966 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470969 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470972 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470975 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470977 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470980 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470983 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470987 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470992 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.470997 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471000 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471003 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471006 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 19:03:57.475437 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471008 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471011 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471013 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471016 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471018 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471020 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471023 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471025 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471028 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471031 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471033 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471036 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471038 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471041 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471043 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471046 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471048 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471051 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471054 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471056 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 19:03:57.475828 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471060 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471064 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471068 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471072 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471075 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471079 2574 feature_gate.go:328] unrecognized feature gate: Example Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471081 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471084 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471088 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471092 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471095 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471098 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471100 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471104 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471107 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471109 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471112 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471114 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471117 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 19:03:57.476326 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471119 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471121 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471124 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471126 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471128 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471131 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471133 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471136 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471139 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471143 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471148 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471152 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471155 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471157 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471161 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471163 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471166 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471168 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471171 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471173 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 19:03:57.476797 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471176 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471179 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471181 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471184 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471187 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471189 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471192 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471195 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471198 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471200 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471202 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471205 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:57.471208 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.471227 2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.471837 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 19:03:57.477300 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.473786 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 19:03:57.477669 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.474531 2574 server.go:1019] "Starting client certificate rotation" Jun 02 19:03:57.477669 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.474631 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 19:03:57.477669 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.475335 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 19:03:57.495036 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.495018 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 19:03:57.496835 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.496815 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 19:03:57.509751 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.509732 2574 log.go:25] "Validated CRI v1 runtime API" Jun 02 19:03:57.515031 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.515016 2574 log.go:25] "Validated CRI v1 image API" Jun 02 19:03:57.516102 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.516078 2574 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 19:03:57.518079 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.518060 2574 fs.go:135] Filesystem UUIDs: map[2faf7376-38c5-449f-ac5e-765be8b14376:/dev/nvme0n1p3 425fe842-7343-41c6-8dc2-9b89466ee417:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jun 02 19:03:57.518153 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.518078 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 19:03:57.522361 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.522346 2574 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 19:03:57.522496 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.522377 2574 manager.go:217] Machine: {Timestamp:2026-06-02 19:03:57.521876342 +0000 UTC m=+0.339846056 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099447 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27bd6b7ce94198440a81bf74b8afbf SystemUUID:ec27bd6b-7ce9-4198-440a-81bf74b8afbf BootID:a51bf520-cd35-4f70-8840-f89ecd98055a Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3a:68:53:3d:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3a:68:53:3d:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:fb:9a:86:79:15 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 19:03:57.523063 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.523053 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 19:03:57.523135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.523128 2574 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 19:03:57.525918 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.525898 2574 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 19:03:57.526039 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.525920 2574 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-102.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 19:03:57.526085 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.526047 2574 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 19:03:57.526085 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.526056 2574 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 19:03:57.526085 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.526063 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 19:03:57.526687 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.526677 2574 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 19:03:57.527513 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.527504 2574 state_mem.go:36] "Initialized new in-memory state store" Jun 02 19:03:57.527618 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.527609 2574 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 19:03:57.529539 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.529529 2574 kubelet.go:491] "Attempting to sync node with API server" Jun 02 19:03:57.529578 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.529541 2574 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 19:03:57.529578 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.529553 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 19:03:57.529578 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.529562 2574 kubelet.go:397] "Adding apiserver pod source" Jun 02 19:03:57.529578 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.529570 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 19:03:57.530919 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.530907 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 19:03:57.530967 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.530928 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 19:03:57.533418 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.533405 2574 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 19:03:57.535475 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.535462 2574 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 19:03:57.537728 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537716 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 19:03:57.537783 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537737 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 19:03:57.537783 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537746 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 19:03:57.537783 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537752 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 19:03:57.537783 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537762 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 19:03:57.537783 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537767 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 19:03:57.537783 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537773 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 19:03:57.537783 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537778 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 19:03:57.537783 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537784 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 19:03:57.537984 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537790 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 19:03:57.537984 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537798 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 19:03:57.537984 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.537813 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 19:03:57.538539 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.538528 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 19:03:57.538575 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.538540 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 19:03:57.540035 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.540000 2574 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-102.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 19:03:57.540098 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.540043 2574 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 19:03:57.540098 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.540063 2574 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 19:03:57.541547 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.541533 2574 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 19:03:57.541613 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.541569 2574 server.go:1295] "Started kubelet" Jun 02 19:03:57.541695 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.541644 2574 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 19:03:57.541789 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.541732 2574 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 19:03:57.541789 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.541646 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 19:03:57.542297 ip-10-0-138-102 systemd[1]: Started Kubernetes Kubelet. Jun 02 19:03:57.545418 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.545397 2574 server.go:317] "Adding debug handlers to kubelet server" Jun 02 19:03:57.545821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.545801 2574 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 19:03:57.545997 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.545890 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j6chr" Jun 02 19:03:57.549732 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.549711 2574 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 19:03:57.549991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.549973 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 19:03:57.550460 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.550443 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 19:03:57.550621 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.550601 2574 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 19:03:57.550621 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.550618 2574 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 19:03:57.550752 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.550702 2574 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 19:03:57.550794 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.550782 2574 reconstruct.go:97] "Volume reconstruction finished" Jun 02 19:03:57.550794 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.550790 2574 reconciler.go:26] "Reconciler: start to sync state" Jun 02 19:03:57.551006 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.550993 2574 factory.go:55] Registering systemd factory Jun 02 19:03:57.551063 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.551032 2574 factory.go:223] Registration of the systemd container factory successfully Jun 02 19:03:57.551241 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.551204 2574 factory.go:153] Registering CRI-O factory Jun 02 19:03:57.551241 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.551239 2574 factory.go:223] Registration of the crio container factory successfully Jun 02 19:03:57.551365 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.551250 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:57.551365 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.551295 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 19:03:57.551365 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.551316 2574 factory.go:103] Registering Raw factory Jun 02 19:03:57.551365 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.551330 2574 manager.go:1196] Started watching for new ooms in manager Jun 02 19:03:57.551923 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.551903 2574 manager.go:319] Starting recovery of all containers Jun 02 19:03:57.554234 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.554201 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j6chr" Jun 02 19:03:57.556940 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.556913 2574 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 02 19:03:57.557615 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.556837 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-102.ec2.internal.18b558f5212a39ae default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-102.ec2.internal,UID:ip-10-0-138-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-102.ec2.internal,},FirstTimestamp:2026-06-02 19:03:57.541546414 +0000 UTC m=+0.359516129,LastTimestamp:2026-06-02 19:03:57.541546414 +0000 UTC m=+0.359516129,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-102.ec2.internal,}" Jun 02 19:03:57.557914 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.557884 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 02 19:03:57.559519 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.559495 2574 manager.go:324] Recovery completed Jun 02 19:03:57.564573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.564485 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:03:57.566148 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.566130 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:03:57.566234 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.566158 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:03:57.566234 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.566168 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:03:57.566627 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.566613 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 19:03:57.566676 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.566627 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 19:03:57.566676 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.566643 2574 state_mem.go:36] "Initialized new in-memory state store" Jun 02 19:03:57.569070 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.569059 2574 policy_none.go:49] "None policy: Start" Jun 02 19:03:57.569107 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.569076 2574 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 19:03:57.569107 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.569086 2574 state_mem.go:35] "Initializing new in-memory state store" Jun 02 19:03:57.614785 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.614770 2574 manager.go:341] "Starting Device Plugin manager" Jun 02 19:03:57.617457 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.614802 2574 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 19:03:57.617457 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.614811 2574 server.go:85] "Starting device plugin registration server" Jun 02 19:03:57.617457 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.615014 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 19:03:57.617457 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.615034 2574 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 19:03:57.617457 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.615154 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 19:03:57.617457 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.615253 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 19:03:57.617457 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.615263 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 19:03:57.617457 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.615715 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 19:03:57.617457 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.615746 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:57.677289 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.677264 2574 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 19:03:57.678402 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.678384 2574 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 19:03:57.678488 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.678407 2574 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 19:03:57.678488 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.678421 2574 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 19:03:57.678488 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.678430 2574 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 19:03:57.678488 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.678468 2574 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 19:03:57.681451 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.681436 2574 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:03:57.715546 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.715507 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:03:57.716242 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.716228 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:03:57.716301 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.716256 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:03:57.716301 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.716268 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:03:57.716301 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.716287 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-102.ec2.internal" Jun 02 19:03:57.724586 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.724572 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-102.ec2.internal" Jun 02 19:03:57.724643 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.724591 2574 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-102.ec2.internal\": node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:57.740710 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.740686 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:57.779439 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.779418 2574 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-102.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal"] Jun 02 19:03:57.779502 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.779473 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:03:57.780154 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.780140 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:03:57.780198 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.780167 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:03:57.780198 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.780179 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:03:57.781330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.781318 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:03:57.781503 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.781490 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-102.ec2.internal" Jun 02 19:03:57.781546 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.781516 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:03:57.781980 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.781957 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:03:57.782067 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.781986 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:03:57.782067 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.782000 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:03:57.782067 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.781964 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:03:57.782067 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.782050 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:03:57.782067 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.782060 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:03:57.783090 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.783073 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" Jun 02 19:03:57.783174 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.783097 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 19:03:57.783709 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.783688 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientMemory" Jun 02 19:03:57.783709 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.783710 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 19:03:57.783808 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.783723 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeHasSufficientPID" Jun 02 19:03:57.810756 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.810732 2574 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-102.ec2.internal\" not found" node="ip-10-0-138-102.ec2.internal" Jun 02 19:03:57.814983 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.814967 2574 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-102.ec2.internal\" not found" node="ip-10-0-138-102.ec2.internal" Jun 02 19:03:57.841429 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.841410 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:57.941818 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:57.941802 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:57.951852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.951835 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e63a79e611900e654aee32f828be8f1b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal\" (UID: \"e63a79e611900e654aee32f828be8f1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" Jun 02 19:03:57.951902 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.951858 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e63a79e611900e654aee32f828be8f1b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal\" (UID: \"e63a79e611900e654aee32f828be8f1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" Jun 02 19:03:57.951902 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:57.951876 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f5ba42d23e87cc5c7afced33d88e7c0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-102.ec2.internal\" (UID: \"9f5ba42d23e87cc5c7afced33d88e7c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-102.ec2.internal" Jun 02 19:03:58.042157 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:58.042116 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:58.052474 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.052454 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e63a79e611900e654aee32f828be8f1b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal\" (UID: \"e63a79e611900e654aee32f828be8f1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" Jun 02 19:03:58.052526 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.052479 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f5ba42d23e87cc5c7afced33d88e7c0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-102.ec2.internal\" (UID: \"9f5ba42d23e87cc5c7afced33d88e7c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-102.ec2.internal" Jun 02 19:03:58.052526 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.052495 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e63a79e611900e654aee32f828be8f1b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal\" (UID: \"e63a79e611900e654aee32f828be8f1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" Jun 02 19:03:58.052628 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.052534 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e63a79e611900e654aee32f828be8f1b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal\" (UID: \"e63a79e611900e654aee32f828be8f1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" Jun 02 19:03:58.052628 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.052551 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e63a79e611900e654aee32f828be8f1b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal\" (UID: \"e63a79e611900e654aee32f828be8f1b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" Jun 02 19:03:58.052628 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.052591 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f5ba42d23e87cc5c7afced33d88e7c0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-102.ec2.internal\" (UID: \"9f5ba42d23e87cc5c7afced33d88e7c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-102.ec2.internal" Jun 02 19:03:58.113610 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.113583 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-102.ec2.internal" Jun 02 19:03:58.117949 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.117929 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" Jun 02 19:03:58.142899 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:58.142873 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:58.243370 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:58.243347 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:58.343855 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:58.343804 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:58.444265 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:58.444247 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:58.474706 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.474688 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 19:03:58.475135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.474830 2574 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 19:03:58.544587 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:58.544568 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:58.550872 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.550856 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 19:03:58.555952 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.555923 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 18:58:57 +0000 UTC" deadline="2028-01-17 06:20:17.919471776 +0000 UTC" Jun 02 19:03:58.556030 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.555952 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14243h16m19.363523699s" Jun 02 19:03:58.559435 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:58.559413 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode63a79e611900e654aee32f828be8f1b.slice/crio-678a76051b51dbce359c6df75457bbea26cf992a2b95087ce405e9434cf6cf32 WatchSource:0}: Error finding container 678a76051b51dbce359c6df75457bbea26cf992a2b95087ce405e9434cf6cf32: Status 404 returned error can't find the container with id 678a76051b51dbce359c6df75457bbea26cf992a2b95087ce405e9434cf6cf32 Jun 02 19:03:58.559603 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:03:58.559591 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f5ba42d23e87cc5c7afced33d88e7c0.slice/crio-4bb3ebce66ce0808d7b6ab90ac658c4a4c82632a43191ac2933d419c0369bd26 WatchSource:0}: Error finding container 4bb3ebce66ce0808d7b6ab90ac658c4a4c82632a43191ac2933d419c0369bd26: Status 404 returned error can't find the container with id 4bb3ebce66ce0808d7b6ab90ac658c4a4c82632a43191ac2933d419c0369bd26 Jun 02 19:03:58.560133 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.560116 2574 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 19:03:58.563368 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.563355 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 19:03:58.582147 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.582125 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zfxgf" Jun 02 19:03:58.589995 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.589977 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zfxgf" Jun 02 19:03:58.644981 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:58.644939 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:58.669832 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.669812 2574 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:03:58.681172 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.681135 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-102.ec2.internal" event={"ID":"9f5ba42d23e87cc5c7afced33d88e7c0","Type":"ContainerStarted","Data":"4bb3ebce66ce0808d7b6ab90ac658c4a4c82632a43191ac2933d419c0369bd26"} Jun 02 19:03:58.682197 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.682174 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" event={"ID":"e63a79e611900e654aee32f828be8f1b","Type":"ContainerStarted","Data":"678a76051b51dbce359c6df75457bbea26cf992a2b95087ce405e9434cf6cf32"} Jun 02 19:03:58.745362 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:58.745342 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:58.845855 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:58.845832 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:58.870014 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:58.869996 2574 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:03:58.946421 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:58.946366 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-102.ec2.internal\" not found" Jun 02 19:03:59.003402 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.003378 2574 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:03:59.051501 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.051481 2574 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-102.ec2.internal" Jun 02 19:03:59.062287 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.062266 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 19:03:59.063063 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.063045 2574 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" Jun 02 19:03:59.073457 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.073440 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 19:03:59.530756 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.530721 2574 apiserver.go:52] "Watching apiserver" Jun 02 19:03:59.537196 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.537172 2574 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 19:03:59.537890 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.537867 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-wrs9f","openshift-multus/multus-zvtrb","openshift-multus/network-metrics-daemon-mp2fh","openshift-ovn-kubernetes/ovnkube-node-4mqnl","kube-system/kube-apiserver-proxy-ip-10-0-138-102.ec2.internal","openshift-dns/node-resolver-dnhhf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal","openshift-multus/multus-additional-cni-plugins-vw2q4","openshift-network-diagnostics/network-check-target-xc29k","openshift-network-operator/iptables-alerter-vd7vd","kube-system/konnectivity-agent-z4qbq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n","openshift-cluster-node-tuning-operator/tuned-ssh8h"] Jun 02 19:03:59.539333 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.539310 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vd7vd" Jun 02 19:03:59.540646 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.540627 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.541440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.541420 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:03:59.541529 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.541458 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 19:03:59.541583 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.541458 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cpmww\"" Jun 02 19:03:59.541639 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.541469 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 19:03:59.541834 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.541813 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:03:59.541937 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:59.541894 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:03:59.542229 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.542200 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 19:03:59.542352 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.542335 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 19:03:59.542460 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.542442 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 19:03:59.542759 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.542743 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 19:03:59.542835 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.542820 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dfzcz\"" Jun 02 19:03:59.543121 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.543101 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wrs9f" Jun 02 19:03:59.544339 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.544316 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.544778 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.544759 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 19:03:59.544862 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.544802 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 19:03:59.544913 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.544859 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 19:03:59.545070 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.545053 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nzqhn\"" Jun 02 19:03:59.545519 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.545501 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dnhhf" Jun 02 19:03:59.546296 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.546064 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 19:03:59.546296 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.546067 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lsm8p\"" Jun 02 19:03:59.546296 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.546070 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:03:59.546759 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.546732 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:03:59.547356 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.547189 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 19:03:59.547356 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.547207 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w4q8m\"" Jun 02 19:03:59.547356 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.547194 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 19:03:59.548782 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.548528 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.548782 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.548533 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 19:03:59.548782 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.548731 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 19:03:59.548782 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.548769 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-j2zm2\"" Jun 02 19:03:59.549876 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.549856 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:03:59.550006 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:59.549929 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:03:59.550198 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.550182 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ldlpd\"" Jun 02 19:03:59.550623 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.550604 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 19:03:59.550856 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.550835 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 19:03:59.552536 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.552514 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.552629 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.552590 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.554226 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.554173 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 19:03:59.554324 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.554293 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 19:03:59.554386 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.554294 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 19:03:59.554615 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.554594 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 19:03:59.554718 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.554702 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 19:03:59.555035 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.554932 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 19:03:59.555035 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.554958 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nfhq6\"" Jun 02 19:03:59.555238 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.555094 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 19:03:59.555305 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.555273 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xkdwx\"" Jun 02 19:03:59.555404 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.555388 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 19:03:59.556723 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.556701 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 19:03:59.559645 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.559623 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ade5e001-4217-4c82-8105-53057c166c59-host-slash\") pod \"iptables-alerter-vd7vd\" (UID: \"ade5e001-4217-4c82-8105-53057c166c59\") " pod="openshift-network-operator/iptables-alerter-vd7vd" Jun 02 19:03:59.559732 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.559674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd8tk\" (UniqueName: \"kubernetes.io/projected/9efd8fa3-600f-42a8-83f5-25d80caacdd7-kube-api-access-rd8tk\") pod \"node-ca-wrs9f\" (UID: \"9efd8fa3-600f-42a8-83f5-25d80caacdd7\") " pod="openshift-image-registry/node-ca-wrs9f" Jun 02 19:03:59.559732 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.559711 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-run-ovn\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.559815 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.559739 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxv9l\" (UniqueName: \"kubernetes.io/projected/12107e6a-8aed-4528-a741-e28f41659026-kube-api-access-sxv9l\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.559815 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.559768 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9efd8fa3-600f-42a8-83f5-25d80caacdd7-host\") pod \"node-ca-wrs9f\" (UID: \"9efd8fa3-600f-42a8-83f5-25d80caacdd7\") " pod="openshift-image-registry/node-ca-wrs9f" Jun 02 19:03:59.559815 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.559797 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-var-lib-kubelet\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.559952 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.559827 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-hostroot\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.559952 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.559855 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-systemd-units\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.560043 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.559948 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.560043 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560010 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-socket-dir-parent\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.560043 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560037 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-cni-netd\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.560188 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560077 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/12107e6a-8aed-4528-a741-e28f41659026-env-overrides\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.560188 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560110 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e9cc9fb-40ad-4f89-923b-eaa811f2a424-konnectivity-ca\") pod \"konnectivity-agent-z4qbq\" (UID: \"3e9cc9fb-40ad-4f89-923b-eaa811f2a424\") " pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:03:59.560298 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560199 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-cni-dir\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.560298 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560254 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-sysconfig\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.560383 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560303 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-kubelet\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.560383 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560374 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-slash\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.560467 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560404 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/12107e6a-8aed-4528-a741-e28f41659026-ovn-node-metrics-cert\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.560467 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560435 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4vr4\" (UniqueName: \"kubernetes.io/projected/6f947012-134a-4905-8242-57a12df7ec9f-kube-api-access-m4vr4\") pod \"node-resolver-dnhhf\" (UID: \"6f947012-134a-4905-8242-57a12df7ec9f\") " pod="openshift-dns/node-resolver-dnhhf" Jun 02 19:03:59.560558 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560466 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-modprobe-d\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.560605 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560592 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-sysctl-conf\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.560945 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560924 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee044637-ed86-4155-b190-3cb2f0d19e00-tmp\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.561023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560954 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6f947012-134a-4905-8242-57a12df7ec9f-hosts-file\") pod \"node-resolver-dnhhf\" (UID: \"6f947012-134a-4905-8242-57a12df7ec9f\") " pod="openshift-dns/node-resolver-dnhhf" Jun 02 19:03:59.561023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560975 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-host\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.561023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.560998 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-tuned\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.561175 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561024 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.561175 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561060 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f947012-134a-4905-8242-57a12df7ec9f-tmp-dir\") pod \"node-resolver-dnhhf\" (UID: \"6f947012-134a-4905-8242-57a12df7ec9f\") " pod="openshift-dns/node-resolver-dnhhf" Jun 02 19:03:59.561175 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561084 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-cnibin\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.561175 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561105 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-os-release\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.561175 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561126 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-daemon-config\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.561175 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561146 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-run-ovn-kubernetes\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.561175 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561162 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6nhn\" (UniqueName: \"kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn\") pod \"network-check-target-xc29k\" (UID: \"ebe8479b-96cc-4fd7-928d-fd02f2345f2a\") " pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:03:59.561175 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561175 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-etc-kubernetes\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.561518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561189 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:03:59.561518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561201 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-sys\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.561518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561235 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-log-socket\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.561518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561260 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-os-release\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.561518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561350 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-run-k8s-cni-cncf-io\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.561518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561394 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-927hq\" (UniqueName: \"kubernetes.io/projected/56b68433-83da-422d-be07-0a4e5c2e1f40-kube-api-access-927hq\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.561518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-cni-bin\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.561518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561443 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/12107e6a-8aed-4528-a741-e28f41659026-ovnkube-config\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.561518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561469 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ade5e001-4217-4c82-8105-53057c166c59-iptables-alerter-script\") pod \"iptables-alerter-vd7vd\" (UID: \"ade5e001-4217-4c82-8105-53057c166c59\") " pod="openshift-network-operator/iptables-alerter-vd7vd" Jun 02 19:03:59.561518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561495 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-var-lib-cni-bin\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.561518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561517 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-var-lib-cni-multus\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561539 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-conf-dir\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561562 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-run-openvswitch\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561689 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/12107e6a-8aed-4528-a741-e28f41659026-ovnkube-script-lib\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561751 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-run-netns\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561790 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-var-lib-openvswitch\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561818 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9efd8fa3-600f-42a8-83f5-25d80caacdd7-serviceca\") pod \"node-ca-wrs9f\" (UID: \"9efd8fa3-600f-42a8-83f5-25d80caacdd7\") " pod="openshift-image-registry/node-ca-wrs9f" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561839 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-kubernetes\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561854 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-sysctl-d\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561873 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-systemd\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561899 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-run\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561934 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-lib-modules\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561959 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e9cc9fb-40ad-4f89-923b-eaa811f2a424-agent-certs\") pod \"konnectivity-agent-z4qbq\" (UID: \"3e9cc9fb-40ad-4f89-923b-eaa811f2a424\") " pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:03:59.561991 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.561978 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tb7c\" (UniqueName: \"kubernetes.io/projected/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-kube-api-access-2tb7c\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562013 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56b68433-83da-422d-be07-0a4e5c2e1f40-cni-binary-copy\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562042 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-var-lib-kubelet\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562066 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-run-systemd\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562089 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-node-log\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562111 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-system-cni-dir\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562136 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562161 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-cnibin\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562184 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9smn8\" (UniqueName: \"kubernetes.io/projected/ee044637-ed86-4155-b190-3cb2f0d19e00-kube-api-access-9smn8\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562208 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-etc-openvswitch\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562254 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562278 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562299 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-run-netns\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562323 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7vsq\" (UniqueName: \"kubernetes.io/projected/ade5e001-4217-4c82-8105-53057c166c59-kube-api-access-v7vsq\") pod \"iptables-alerter-vd7vd\" (UID: \"ade5e001-4217-4c82-8105-53057c166c59\") " pod="openshift-network-operator/iptables-alerter-vd7vd" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562344 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-system-cni-dir\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562368 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-run-multus-certs\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.562573 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.562392 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hqlp\" (UniqueName: \"kubernetes.io/projected/9e006346-8709-4a02-bf44-d9e5a59bb74b-kube-api-access-2hqlp\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:03:59.591075 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.591054 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 18:58:58 +0000 UTC" deadline="2027-11-22 16:16:17.987498891 +0000 UTC" Jun 02 19:03:59.591075 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.591074 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12909h12m18.396427528s" Jun 02 19:03:59.652252 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.652229 2574 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 19:03:59.663022 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.662984 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e9cc9fb-40ad-4f89-923b-eaa811f2a424-konnectivity-ca\") pod \"konnectivity-agent-z4qbq\" (UID: \"3e9cc9fb-40ad-4f89-923b-eaa811f2a424\") " pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:03:59.663127 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663027 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-cni-dir\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.663127 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663044 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-sysconfig\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.663127 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663066 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-kubelet\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.663127 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663089 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-slash\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.663127 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663112 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/12107e6a-8aed-4528-a741-e28f41659026-ovn-node-metrics-cert\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.663127 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663121 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-sysconfig\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663136 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m4vr4\" (UniqueName: \"kubernetes.io/projected/6f947012-134a-4905-8242-57a12df7ec9f-kube-api-access-m4vr4\") pod \"node-resolver-dnhhf\" (UID: \"6f947012-134a-4905-8242-57a12df7ec9f\") " pod="openshift-dns/node-resolver-dnhhf" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663122 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-kubelet\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663145 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-slash\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663159 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-modprobe-d\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663189 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-cni-dir\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663203 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-sysctl-conf\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663250 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee044637-ed86-4155-b190-3cb2f0d19e00-tmp\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663276 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6f947012-134a-4905-8242-57a12df7ec9f-hosts-file\") pod \"node-resolver-dnhhf\" (UID: \"6f947012-134a-4905-8242-57a12df7ec9f\") " pod="openshift-dns/node-resolver-dnhhf" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663279 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-modprobe-d\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663327 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6f947012-134a-4905-8242-57a12df7ec9f-hosts-file\") pod \"node-resolver-dnhhf\" (UID: \"6f947012-134a-4905-8242-57a12df7ec9f\") " pod="openshift-dns/node-resolver-dnhhf" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663361 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-host\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663387 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-tuned\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663413 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663418 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-host\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663383 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-sysctl-conf\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.663440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663442 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f947012-134a-4905-8242-57a12df7ec9f-tmp-dir\") pod \"node-resolver-dnhhf\" (UID: \"6f947012-134a-4905-8242-57a12df7ec9f\") " pod="openshift-dns/node-resolver-dnhhf" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663465 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-cnibin\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663468 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663493 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-os-release\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663492 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663530 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-daemon-config\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663543 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-cnibin\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663559 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-run-ovn-kubernetes\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663588 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-device-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663592 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e9cc9fb-40ad-4f89-923b-eaa811f2a424-konnectivity-ca\") pod \"konnectivity-agent-z4qbq\" (UID: \"3e9cc9fb-40ad-4f89-923b-eaa811f2a424\") " pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663613 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-sys-fs\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663642 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nhn\" (UniqueName: \"kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn\") pod \"network-check-target-xc29k\" (UID: \"ebe8479b-96cc-4fd7-928d-fd02f2345f2a\") " pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663647 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-run-ovn-kubernetes\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663666 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-etc-kubernetes\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663695 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663702 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-etc-kubernetes\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663719 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-sys\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663595 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-os-release\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.664330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663760 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f947012-134a-4905-8242-57a12df7ec9f-tmp-dir\") pod \"node-resolver-dnhhf\" (UID: \"6f947012-134a-4905-8242-57a12df7ec9f\") " pod="openshift-dns/node-resolver-dnhhf" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663765 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-sys\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:59.663831 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663857 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-log-socket\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663887 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-os-release\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:59.663916 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs podName:9e006346-8709-4a02-bf44-d9e5a59bb74b nodeName:}" failed. No retries permitted until 2026-06-02 19:04:00.163881827 +0000 UTC m=+2.981851552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs") pod "network-metrics-daemon-mp2fh" (UID: "9e006346-8709-4a02-bf44-d9e5a59bb74b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663939 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-os-release\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663945 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-run-k8s-cni-cncf-io\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663975 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-927hq\" (UniqueName: \"kubernetes.io/projected/56b68433-83da-422d-be07-0a4e5c2e1f40-kube-api-access-927hq\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.663979 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-run-k8s-cni-cncf-io\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664009 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-cni-bin\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664010 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-log-socket\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664042 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-cni-bin\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664042 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/12107e6a-8aed-4528-a741-e28f41659026-ovnkube-config\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664076 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-registration-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664099 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x4d5\" (UniqueName: \"kubernetes.io/projected/e61262b3-9ef8-43ec-b211-15c001166361-kube-api-access-8x4d5\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664122 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ade5e001-4217-4c82-8105-53057c166c59-iptables-alerter-script\") pod \"iptables-alerter-vd7vd\" (UID: \"ade5e001-4217-4c82-8105-53057c166c59\") " pod="openshift-network-operator/iptables-alerter-vd7vd" Jun 02 19:03:59.665142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664142 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-var-lib-cni-bin\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664153 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-daemon-config\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664163 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-var-lib-cni-multus\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664198 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-conf-dir\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664243 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-run-openvswitch\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664264 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-conf-dir\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664269 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/12107e6a-8aed-4528-a741-e28f41659026-ovnkube-script-lib\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664271 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-var-lib-cni-bin\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664325 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-run-netns\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664352 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-var-lib-openvswitch\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664393 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-run-netns\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664429 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-var-lib-openvswitch\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664462 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9efd8fa3-600f-42a8-83f5-25d80caacdd7-serviceca\") pod \"node-ca-wrs9f\" (UID: \"9efd8fa3-600f-42a8-83f5-25d80caacdd7\") " pod="openshift-image-registry/node-ca-wrs9f" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664498 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-kubernetes\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664525 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-sysctl-d\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664552 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-systemd\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664577 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-run\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664605 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/12107e6a-8aed-4528-a741-e28f41659026-ovnkube-config\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.666023 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664630 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-lib-modules\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664655 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e9cc9fb-40ad-4f89-923b-eaa811f2a424-agent-certs\") pod \"konnectivity-agent-z4qbq\" (UID: \"3e9cc9fb-40ad-4f89-923b-eaa811f2a424\") " pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664684 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2tb7c\" (UniqueName: \"kubernetes.io/projected/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-kube-api-access-2tb7c\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664707 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56b68433-83da-422d-be07-0a4e5c2e1f40-cni-binary-copy\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664728 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-var-lib-kubelet\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664749 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-run-systemd\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664772 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-node-log\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664802 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664843 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ade5e001-4217-4c82-8105-53057c166c59-iptables-alerter-script\") pod \"iptables-alerter-vd7vd\" (UID: \"ade5e001-4217-4c82-8105-53057c166c59\") " pod="openshift-network-operator/iptables-alerter-vd7vd" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664872 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-system-cni-dir\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664879 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/12107e6a-8aed-4528-a741-e28f41659026-ovnkube-script-lib\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664882 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-var-lib-kubelet\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664900 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664926 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-cnibin\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664948 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-run-openvswitch\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664952 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-lib-modules\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664950 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9smn8\" (UniqueName: \"kubernetes.io/projected/ee044637-ed86-4155-b190-3cb2f0d19e00-kube-api-access-9smn8\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.666852 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.664978 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-var-lib-cni-multus\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665018 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-node-log\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665026 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-run-systemd\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665139 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-sysctl-d\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665184 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-run\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665244 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-kubernetes\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665281 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-system-cni-dir\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665322 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-systemd\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665368 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-cnibin\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665411 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56b68433-83da-422d-be07-0a4e5c2e1f40-cni-binary-copy\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665447 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-etc-openvswitch\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665479 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665507 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665516 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665531 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-run-netns\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665541 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-etc-openvswitch\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665550 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9efd8fa3-600f-42a8-83f5-25d80caacdd7-serviceca\") pod \"node-ca-wrs9f\" (UID: \"9efd8fa3-600f-42a8-83f5-25d80caacdd7\") " pod="openshift-image-registry/node-ca-wrs9f" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665580 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-run-netns\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.667679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665591 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-etc-selinux\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665621 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v7vsq\" (UniqueName: \"kubernetes.io/projected/ade5e001-4217-4c82-8105-53057c166c59-kube-api-access-v7vsq\") pod \"iptables-alerter-vd7vd\" (UID: \"ade5e001-4217-4c82-8105-53057c166c59\") " pod="openshift-network-operator/iptables-alerter-vd7vd" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665659 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665697 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-system-cni-dir\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665727 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-run-multus-certs\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665781 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2hqlp\" (UniqueName: \"kubernetes.io/projected/9e006346-8709-4a02-bf44-d9e5a59bb74b-kube-api-access-2hqlp\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665805 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-system-cni-dir\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665808 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ade5e001-4217-4c82-8105-53057c166c59-host-slash\") pod \"iptables-alerter-vd7vd\" (UID: \"ade5e001-4217-4c82-8105-53057c166c59\") " pod="openshift-network-operator/iptables-alerter-vd7vd" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665839 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ade5e001-4217-4c82-8105-53057c166c59-host-slash\") pod \"iptables-alerter-vd7vd\" (UID: \"ade5e001-4217-4c82-8105-53057c166c59\") " pod="openshift-network-operator/iptables-alerter-vd7vd" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665849 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-run-multus-certs\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665853 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rd8tk\" (UniqueName: \"kubernetes.io/projected/9efd8fa3-600f-42a8-83f5-25d80caacdd7-kube-api-access-rd8tk\") pod \"node-ca-wrs9f\" (UID: \"9efd8fa3-600f-42a8-83f5-25d80caacdd7\") " pod="openshift-image-registry/node-ca-wrs9f" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665969 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-run-ovn\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.665998 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sxv9l\" (UniqueName: \"kubernetes.io/projected/12107e6a-8aed-4528-a741-e28f41659026-kube-api-access-sxv9l\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666035 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9efd8fa3-600f-42a8-83f5-25d80caacdd7-host\") pod \"node-ca-wrs9f\" (UID: \"9efd8fa3-600f-42a8-83f5-25d80caacdd7\") " pod="openshift-image-registry/node-ca-wrs9f" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666062 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-var-lib-kubelet\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666065 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-run-ovn\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666099 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9efd8fa3-600f-42a8-83f5-25d80caacdd7-host\") pod \"node-ca-wrs9f\" (UID: \"9efd8fa3-600f-42a8-83f5-25d80caacdd7\") " pod="openshift-image-registry/node-ca-wrs9f" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666119 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-host-var-lib-kubelet\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.668534 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666131 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-hostroot\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666159 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-systemd-units\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666184 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666211 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-socket-dir-parent\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666253 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-cni-netd\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666258 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-hostroot\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666279 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/12107e6a-8aed-4528-a741-e28f41659026-env-overrides\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666304 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-host-cni-netd\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666309 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-socket-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666355 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56b68433-83da-422d-be07-0a4e5c2e1f40-multus-socket-dir-parent\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666252 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/12107e6a-8aed-4528-a741-e28f41659026-systemd-units\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666778 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666806 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/12107e6a-8aed-4528-a741-e28f41659026-env-overrides\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.666988 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee044637-ed86-4155-b190-3cb2f0d19e00-etc-tuned\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.667536 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.667584 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee044637-ed86-4155-b190-3cb2f0d19e00-tmp\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.667948 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/12107e6a-8aed-4528-a741-e28f41659026-ovn-node-metrics-cert\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.669363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.668461 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e9cc9fb-40ad-4f89-923b-eaa811f2a424-agent-certs\") pod \"konnectivity-agent-z4qbq\" (UID: \"3e9cc9fb-40ad-4f89-923b-eaa811f2a424\") " pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:03:59.675210 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:59.675184 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:03:59.675304 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:59.675212 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:03:59.675304 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:59.675247 2574 projected.go:194] Error preparing data for projected volume kube-api-access-k6nhn for pod openshift-network-diagnostics/network-check-target-xc29k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:03:59.676249 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:03:59.676229 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn podName:ebe8479b-96cc-4fd7-928d-fd02f2345f2a nodeName:}" failed. No retries permitted until 2026-06-02 19:04:00.176179871 +0000 UTC m=+2.994149587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k6nhn" (UniqueName: "kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn") pod "network-check-target-xc29k" (UID: "ebe8479b-96cc-4fd7-928d-fd02f2345f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:03:59.676804 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.676782 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4vr4\" (UniqueName: \"kubernetes.io/projected/6f947012-134a-4905-8242-57a12df7ec9f-kube-api-access-m4vr4\") pod \"node-resolver-dnhhf\" (UID: \"6f947012-134a-4905-8242-57a12df7ec9f\") " pod="openshift-dns/node-resolver-dnhhf" Jun 02 19:03:59.676892 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.676863 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-927hq\" (UniqueName: \"kubernetes.io/projected/56b68433-83da-422d-be07-0a4e5c2e1f40-kube-api-access-927hq\") pod \"multus-zvtrb\" (UID: \"56b68433-83da-422d-be07-0a4e5c2e1f40\") " pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.677446 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.677393 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9smn8\" (UniqueName: \"kubernetes.io/projected/ee044637-ed86-4155-b190-3cb2f0d19e00-kube-api-access-9smn8\") pod \"tuned-ssh8h\" (UID: \"ee044637-ed86-4155-b190-3cb2f0d19e00\") " pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.678960 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.678936 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tb7c\" (UniqueName: \"kubernetes.io/projected/ee8b07c2-5feb-4d66-87aa-aee5c65834dd-kube-api-access-2tb7c\") pod \"multus-additional-cni-plugins-vw2q4\" (UID: \"ee8b07c2-5feb-4d66-87aa-aee5c65834dd\") " pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.679875 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.679627 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7vsq\" (UniqueName: \"kubernetes.io/projected/ade5e001-4217-4c82-8105-53057c166c59-kube-api-access-v7vsq\") pod \"iptables-alerter-vd7vd\" (UID: \"ade5e001-4217-4c82-8105-53057c166c59\") " pod="openshift-network-operator/iptables-alerter-vd7vd" Jun 02 19:03:59.679875 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.679851 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hqlp\" (UniqueName: \"kubernetes.io/projected/9e006346-8709-4a02-bf44-d9e5a59bb74b-kube-api-access-2hqlp\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:03:59.684120 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.684098 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd8tk\" (UniqueName: \"kubernetes.io/projected/9efd8fa3-600f-42a8-83f5-25d80caacdd7-kube-api-access-rd8tk\") pod \"node-ca-wrs9f\" (UID: \"9efd8fa3-600f-42a8-83f5-25d80caacdd7\") " pod="openshift-image-registry/node-ca-wrs9f" Jun 02 19:03:59.686143 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.686120 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxv9l\" (UniqueName: \"kubernetes.io/projected/12107e6a-8aed-4528-a741-e28f41659026-kube-api-access-sxv9l\") pod \"ovnkube-node-4mqnl\" (UID: \"12107e6a-8aed-4528-a741-e28f41659026\") " pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:03:59.767600 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767579 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-device-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767737 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767606 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-sys-fs\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767737 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767654 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-registration-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767737 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767677 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8x4d5\" (UniqueName: \"kubernetes.io/projected/e61262b3-9ef8-43ec-b211-15c001166361-kube-api-access-8x4d5\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767737 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767707 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767737 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767705 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-device-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767737 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767720 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-sys-fs\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767737 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767736 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-etc-selinux\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767995 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767761 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-socket-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767995 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767769 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-registration-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767995 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767783 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767995 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767876 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-etc-selinux\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.767995 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.767879 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e61262b3-9ef8-43ec-b211-15c001166361-socket-dir\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.781645 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.781601 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x4d5\" (UniqueName: \"kubernetes.io/projected/e61262b3-9ef8-43ec-b211-15c001166361-kube-api-access-8x4d5\") pod \"aws-ebs-csi-driver-node-gnm2n\" (UID: \"e61262b3-9ef8-43ec-b211-15c001166361\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.848907 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.848882 2574 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 19:03:59.852162 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.852141 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vd7vd" Jun 02 19:03:59.859813 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.859792 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zvtrb" Jun 02 19:03:59.868305 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.868283 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wrs9f" Jun 02 19:03:59.872890 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.872874 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" Jun 02 19:03:59.880347 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.880330 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dnhhf" Jun 02 19:03:59.886887 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.886872 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:03:59.893449 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.893433 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vw2q4" Jun 02 19:03:59.900980 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.900963 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" Jun 02 19:03:59.905527 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:03:59.905509 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:04:00.148501 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:00.148471 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f947012_134a_4905_8242_57a12df7ec9f.slice/crio-a9ce38ce27cfa5a42f5a82b5d25ef252ab2da77b315c9e80f0ba3c7668f0e828 WatchSource:0}: Error finding container a9ce38ce27cfa5a42f5a82b5d25ef252ab2da77b315c9e80f0ba3c7668f0e828: Status 404 returned error can't find the container with id a9ce38ce27cfa5a42f5a82b5d25ef252ab2da77b315c9e80f0ba3c7668f0e828 Jun 02 19:04:00.150066 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:00.149983 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode61262b3_9ef8_43ec_b211_15c001166361.slice/crio-fb72897bb888a34e8d38242e5d84d118abfc3be1f367929a2512550935e76eb7 WatchSource:0}: Error finding container fb72897bb888a34e8d38242e5d84d118abfc3be1f367929a2512550935e76eb7: Status 404 returned error can't find the container with id fb72897bb888a34e8d38242e5d84d118abfc3be1f367929a2512550935e76eb7 Jun 02 19:04:00.150244 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:00.150209 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efd8fa3_600f_42a8_83f5_25d80caacdd7.slice/crio-d1d635099fc8fdbeb5c226e56e7e9b3da5e0650d929f57a551039a7c5bd08bc6 WatchSource:0}: Error finding container d1d635099fc8fdbeb5c226e56e7e9b3da5e0650d929f57a551039a7c5bd08bc6: Status 404 returned error can't find the container with id d1d635099fc8fdbeb5c226e56e7e9b3da5e0650d929f57a551039a7c5bd08bc6 Jun 02 19:04:00.153787 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:00.153748 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9cc9fb_40ad_4f89_923b_eaa811f2a424.slice/crio-6b7c2148431d265ea97867ed079eeae600a0183952b204fa4117e228d3fbaa60 WatchSource:0}: Error finding container 6b7c2148431d265ea97867ed079eeae600a0183952b204fa4117e228d3fbaa60: Status 404 returned error can't find the container with id 6b7c2148431d265ea97867ed079eeae600a0183952b204fa4117e228d3fbaa60 Jun 02 19:04:00.154639 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:00.154614 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee044637_ed86_4155_b190_3cb2f0d19e00.slice/crio-3e4a3fcbd93b69952b4f49efe49f1f93a9f1068894c989bc014ae336e51ba08b WatchSource:0}: Error finding container 3e4a3fcbd93b69952b4f49efe49f1f93a9f1068894c989bc014ae336e51ba08b: Status 404 returned error can't find the container with id 3e4a3fcbd93b69952b4f49efe49f1f93a9f1068894c989bc014ae336e51ba08b Jun 02 19:04:00.155368 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:00.155348 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade5e001_4217_4c82_8105_53057c166c59.slice/crio-0df0626be063ff2e80838f20f306a9041b8004fb24034e7db11c434b18b41806 WatchSource:0}: Error finding container 0df0626be063ff2e80838f20f306a9041b8004fb24034e7db11c434b18b41806: Status 404 returned error can't find the container with id 0df0626be063ff2e80838f20f306a9041b8004fb24034e7db11c434b18b41806 Jun 02 19:04:00.156330 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:00.156299 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8b07c2_5feb_4d66_87aa_aee5c65834dd.slice/crio-84094735dde4b14f7d976935e7591ea29743e94fb16041edbf44833fba272107 WatchSource:0}: Error finding container 84094735dde4b14f7d976935e7591ea29743e94fb16041edbf44833fba272107: Status 404 returned error can't find the container with id 84094735dde4b14f7d976935e7591ea29743e94fb16041edbf44833fba272107 Jun 02 19:04:00.158922 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:00.158896 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b68433_83da_422d_be07_0a4e5c2e1f40.slice/crio-d93ea5ba2d1cc0be04dec95a5706671a98ede607ba1dc6b41b6fd1ce4b6e6cd6 WatchSource:0}: Error finding container d93ea5ba2d1cc0be04dec95a5706671a98ede607ba1dc6b41b6fd1ce4b6e6cd6: Status 404 returned error can't find the container with id d93ea5ba2d1cc0be04dec95a5706671a98ede607ba1dc6b41b6fd1ce4b6e6cd6 Jun 02 19:04:00.159937 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:00.159908 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12107e6a_8aed_4528_a741_e28f41659026.slice/crio-04bf49e21d179b3b9d016c72bf6168113439c54fc9be054d16b339b63cdb6e29 WatchSource:0}: Error finding container 04bf49e21d179b3b9d016c72bf6168113439c54fc9be054d16b339b63cdb6e29: Status 404 returned error can't find the container with id 04bf49e21d179b3b9d016c72bf6168113439c54fc9be054d16b339b63cdb6e29 Jun 02 19:04:00.170532 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.170512 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:00.170620 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:00.170609 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:00.170677 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:00.170654 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs podName:9e006346-8709-4a02-bf44-d9e5a59bb74b nodeName:}" failed. No retries permitted until 2026-06-02 19:04:01.170638977 +0000 UTC m=+3.988608701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs") pod "network-metrics-daemon-mp2fh" (UID: "9e006346-8709-4a02-bf44-d9e5a59bb74b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:00.271034 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.270918 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nhn\" (UniqueName: \"kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn\") pod \"network-check-target-xc29k\" (UID: \"ebe8479b-96cc-4fd7-928d-fd02f2345f2a\") " pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:00.271121 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:00.271062 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:04:00.271121 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:00.271080 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:04:00.271121 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:00.271089 2574 projected.go:194] Error preparing data for projected volume kube-api-access-k6nhn for pod openshift-network-diagnostics/network-check-target-xc29k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:00.271240 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:00.271130 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn podName:ebe8479b-96cc-4fd7-928d-fd02f2345f2a nodeName:}" failed. No retries permitted until 2026-06-02 19:04:01.27111685 +0000 UTC m=+4.089086552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6nhn" (UniqueName: "kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn") pod "network-check-target-xc29k" (UID: "ebe8479b-96cc-4fd7-928d-fd02f2345f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:00.591365 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.591326 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 18:58:58 +0000 UTC" deadline="2027-12-29 07:10:37.437531676 +0000 UTC" Jun 02 19:04:00.591365 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.591362 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13788h6m36.846173333s" Jun 02 19:04:00.688059 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.687208 2574 generic.go:358] "Generic (PLEG): container finished" podID="e63a79e611900e654aee32f828be8f1b" containerID="08a586fa19d36ae0f77d0db08d8c32d437e9bc102006afbab35a1ca9c24c5ef8" exitCode=0 Jun 02 19:04:00.688059 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.687319 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" event={"ID":"e63a79e611900e654aee32f828be8f1b","Type":"ContainerDied","Data":"08a586fa19d36ae0f77d0db08d8c32d437e9bc102006afbab35a1ca9c24c5ef8"} Jun 02 19:04:00.692895 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.692835 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zvtrb" event={"ID":"56b68433-83da-422d-be07-0a4e5c2e1f40","Type":"ContainerStarted","Data":"d93ea5ba2d1cc0be04dec95a5706671a98ede607ba1dc6b41b6fd1ce4b6e6cd6"} Jun 02 19:04:00.696341 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.696273 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vd7vd" event={"ID":"ade5e001-4217-4c82-8105-53057c166c59","Type":"ContainerStarted","Data":"0df0626be063ff2e80838f20f306a9041b8004fb24034e7db11c434b18b41806"} Jun 02 19:04:00.697997 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.697973 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" event={"ID":"e61262b3-9ef8-43ec-b211-15c001166361","Type":"ContainerStarted","Data":"fb72897bb888a34e8d38242e5d84d118abfc3be1f367929a2512550935e76eb7"} Jun 02 19:04:00.707929 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.707871 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dnhhf" event={"ID":"6f947012-134a-4905-8242-57a12df7ec9f","Type":"ContainerStarted","Data":"a9ce38ce27cfa5a42f5a82b5d25ef252ab2da77b315c9e80f0ba3c7668f0e828"} Jun 02 19:04:00.709843 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.709746 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" event={"ID":"12107e6a-8aed-4528-a741-e28f41659026","Type":"ContainerStarted","Data":"04bf49e21d179b3b9d016c72bf6168113439c54fc9be054d16b339b63cdb6e29"} Jun 02 19:04:00.713093 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.713032 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" event={"ID":"ee044637-ed86-4155-b190-3cb2f0d19e00","Type":"ContainerStarted","Data":"3e4a3fcbd93b69952b4f49efe49f1f93a9f1068894c989bc014ae336e51ba08b"} Jun 02 19:04:00.715452 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.715398 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2q4" event={"ID":"ee8b07c2-5feb-4d66-87aa-aee5c65834dd","Type":"ContainerStarted","Data":"84094735dde4b14f7d976935e7591ea29743e94fb16041edbf44833fba272107"} Jun 02 19:04:00.716966 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.716906 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z4qbq" event={"ID":"3e9cc9fb-40ad-4f89-923b-eaa811f2a424","Type":"ContainerStarted","Data":"6b7c2148431d265ea97867ed079eeae600a0183952b204fa4117e228d3fbaa60"} Jun 02 19:04:00.718354 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.718312 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wrs9f" event={"ID":"9efd8fa3-600f-42a8-83f5-25d80caacdd7","Type":"ContainerStarted","Data":"d1d635099fc8fdbeb5c226e56e7e9b3da5e0650d929f57a551039a7c5bd08bc6"} Jun 02 19:04:00.722500 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:00.722343 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-102.ec2.internal" event={"ID":"9f5ba42d23e87cc5c7afced33d88e7c0","Type":"ContainerStarted","Data":"98e9013513c9dbc2b3d4bf7f0d22a1a406a7196fcd7397f5eb8b902327f94e26"} Jun 02 19:04:01.177954 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:01.177359 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:01.177954 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:01.177537 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:01.177954 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:01.177597 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs podName:9e006346-8709-4a02-bf44-d9e5a59bb74b nodeName:}" failed. No retries permitted until 2026-06-02 19:04:03.17757835 +0000 UTC m=+5.995548057 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs") pod "network-metrics-daemon-mp2fh" (UID: "9e006346-8709-4a02-bf44-d9e5a59bb74b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:01.278692 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:01.278595 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nhn\" (UniqueName: \"kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn\") pod \"network-check-target-xc29k\" (UID: \"ebe8479b-96cc-4fd7-928d-fd02f2345f2a\") " pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:01.278877 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:01.278778 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:04:01.278877 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:01.278797 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:04:01.278877 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:01.278810 2574 projected.go:194] Error preparing data for projected volume kube-api-access-k6nhn for pod openshift-network-diagnostics/network-check-target-xc29k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:01.278877 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:01.278867 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn podName:ebe8479b-96cc-4fd7-928d-fd02f2345f2a nodeName:}" failed. No retries permitted until 2026-06-02 19:04:03.278847698 +0000 UTC m=+6.096817405 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6nhn" (UniqueName: "kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn") pod "network-check-target-xc29k" (UID: "ebe8479b-96cc-4fd7-928d-fd02f2345f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:01.681856 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:01.681148 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:01.681856 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:01.681298 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:01.681856 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:01.681704 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:01.681856 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:01.681800 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:01.749513 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:01.749451 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" event={"ID":"e63a79e611900e654aee32f828be8f1b","Type":"ContainerStarted","Data":"2a302ff35ce8f15638316830789898f1f7b67964acf56e4c713d76f3b317ebfd"} Jun 02 19:04:01.773894 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:01.773644 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-102.ec2.internal" podStartSLOduration=2.773627741 podStartE2EDuration="2.773627741s" podCreationTimestamp="2026-06-02 19:03:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:04:00.736199109 +0000 UTC m=+3.554168834" watchObservedRunningTime="2026-06-02 19:04:01.773627741 +0000 UTC m=+4.591597467" Jun 02 19:04:03.194479 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:03.194441 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:03.195044 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:03.194610 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:03.195044 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:03.194673 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs podName:9e006346-8709-4a02-bf44-d9e5a59bb74b nodeName:}" failed. No retries permitted until 2026-06-02 19:04:07.194654137 +0000 UTC m=+10.012623847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs") pod "network-metrics-daemon-mp2fh" (UID: "9e006346-8709-4a02-bf44-d9e5a59bb74b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:03.295618 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:03.295582 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nhn\" (UniqueName: \"kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn\") pod \"network-check-target-xc29k\" (UID: \"ebe8479b-96cc-4fd7-928d-fd02f2345f2a\") " pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:03.295775 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:03.295757 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:04:03.295775 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:03.295777 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:04:03.295910 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:03.295789 2574 projected.go:194] Error preparing data for projected volume kube-api-access-k6nhn for pod openshift-network-diagnostics/network-check-target-xc29k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:03.295910 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:03.295847 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn podName:ebe8479b-96cc-4fd7-928d-fd02f2345f2a nodeName:}" failed. No retries permitted until 2026-06-02 19:04:07.295829039 +0000 UTC m=+10.113798758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6nhn" (UniqueName: "kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn") pod "network-check-target-xc29k" (UID: "ebe8479b-96cc-4fd7-928d-fd02f2345f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:03.678980 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:03.678950 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:03.679173 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:03.679076 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:03.679260 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:03.679162 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:03.679339 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:03.679312 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:05.682575 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:05.682544 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:05.682978 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:05.682692 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:05.682978 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:05.682687 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:05.682978 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:05.682762 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:07.223160 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:07.223121 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:07.223614 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:07.223287 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:07.223614 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:07.223366 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs podName:9e006346-8709-4a02-bf44-d9e5a59bb74b nodeName:}" failed. No retries permitted until 2026-06-02 19:04:15.223343455 +0000 UTC m=+18.041313159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs") pod "network-metrics-daemon-mp2fh" (UID: "9e006346-8709-4a02-bf44-d9e5a59bb74b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:07.324330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:07.324296 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nhn\" (UniqueName: \"kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn\") pod \"network-check-target-xc29k\" (UID: \"ebe8479b-96cc-4fd7-928d-fd02f2345f2a\") " pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:07.324494 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:07.324475 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:04:07.324570 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:07.324499 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:04:07.324570 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:07.324512 2574 projected.go:194] Error preparing data for projected volume kube-api-access-k6nhn for pod openshift-network-diagnostics/network-check-target-xc29k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:07.324673 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:07.324575 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn podName:ebe8479b-96cc-4fd7-928d-fd02f2345f2a nodeName:}" failed. No retries permitted until 2026-06-02 19:04:15.324554957 +0000 UTC m=+18.142524663 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6nhn" (UniqueName: "kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn") pod "network-check-target-xc29k" (UID: "ebe8479b-96cc-4fd7-928d-fd02f2345f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:07.681642 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:07.681613 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:07.681642 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:07.681639 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:07.681864 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:07.681721 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:07.682093 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:07.682060 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:09.684478 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:09.684383 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:09.685032 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:09.684511 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:09.685032 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:09.684662 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:09.685032 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:09.684764 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:11.679341 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:11.679314 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:11.679734 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:11.679333 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:11.679734 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:11.679419 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:11.679734 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:11.679560 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:13.678758 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:13.678724 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:13.679210 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:13.678792 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:13.679210 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:13.678897 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:13.679342 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:13.679292 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:15.284998 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:15.284965 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:15.285408 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:15.285068 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:15.285408 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:15.285126 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs podName:9e006346-8709-4a02-bf44-d9e5a59bb74b nodeName:}" failed. No retries permitted until 2026-06-02 19:04:31.285107058 +0000 UTC m=+34.103076761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs") pod "network-metrics-daemon-mp2fh" (UID: "9e006346-8709-4a02-bf44-d9e5a59bb74b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:15.385962 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:15.385930 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nhn\" (UniqueName: \"kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn\") pod \"network-check-target-xc29k\" (UID: \"ebe8479b-96cc-4fd7-928d-fd02f2345f2a\") " pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:15.386129 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:15.386100 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:04:15.386129 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:15.386127 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:04:15.386301 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:15.386140 2574 projected.go:194] Error preparing data for projected volume kube-api-access-k6nhn for pod openshift-network-diagnostics/network-check-target-xc29k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:15.386301 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:15.386188 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn podName:ebe8479b-96cc-4fd7-928d-fd02f2345f2a nodeName:}" failed. No retries permitted until 2026-06-02 19:04:31.386175934 +0000 UTC m=+34.204145636 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6nhn" (UniqueName: "kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn") pod "network-check-target-xc29k" (UID: "ebe8479b-96cc-4fd7-928d-fd02f2345f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:15.679370 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:15.679331 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:15.679531 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:15.679490 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:15.679592 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:15.679554 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:15.679730 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:15.679706 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:17.679543 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:17.679512 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:17.680004 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:17.679590 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:17.680004 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:17.679671 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:17.680004 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:17.679759 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:18.781861 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.781654 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" event={"ID":"12107e6a-8aed-4528-a741-e28f41659026","Type":"ContainerStarted","Data":"1b833eb809aade02d4323215a0b58cbdec0e07ae11acc389e7d5f65d11556d7f"} Jun 02 19:04:18.782520 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.781863 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" event={"ID":"12107e6a-8aed-4528-a741-e28f41659026","Type":"ContainerStarted","Data":"018a2711e83b5c9e8e8a584ba1a1af2ed396818d63135f9251f3f6f2b75e46d3"} Jun 02 19:04:18.782520 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.781881 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" event={"ID":"12107e6a-8aed-4528-a741-e28f41659026","Type":"ContainerStarted","Data":"cbf21e6fc387f9fcda5024262a8f493151f96e0c791c77602ec41a2a84af7271"} Jun 02 19:04:18.782520 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.781893 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" event={"ID":"12107e6a-8aed-4528-a741-e28f41659026","Type":"ContainerStarted","Data":"6434d7c5019b204ccf2ad22695bfba7804dc8588a3b9233aaa4124f2226eaf86"} Jun 02 19:04:18.782520 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.781904 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" event={"ID":"12107e6a-8aed-4528-a741-e28f41659026","Type":"ContainerStarted","Data":"2427663162a22d105b3dc084abc7e8e24748877405cd9edc3f3e4418f1267fb9"} Jun 02 19:04:18.782520 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.781948 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" event={"ID":"12107e6a-8aed-4528-a741-e28f41659026","Type":"ContainerStarted","Data":"68ceca256d5bc3a3c0962c7d8a85d1e04203feb563faa8079f36ab71c5e29d51"} Jun 02 19:04:18.783006 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.782969 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" event={"ID":"ee044637-ed86-4155-b190-3cb2f0d19e00","Type":"ContainerStarted","Data":"34f36e56883b1e7bb67466f245ab554adbe334e034baf1b7c7fcf9dfe337ce32"} Jun 02 19:04:18.784631 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.784607 2574 generic.go:358] "Generic (PLEG): container finished" podID="ee8b07c2-5feb-4d66-87aa-aee5c65834dd" containerID="0cf424882a96bc8a209af123cebe579519195f6d5b3394d62b0b5dbf4fa837fe" exitCode=0 Jun 02 19:04:18.784884 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.784858 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2q4" event={"ID":"ee8b07c2-5feb-4d66-87aa-aee5c65834dd","Type":"ContainerDied","Data":"0cf424882a96bc8a209af123cebe579519195f6d5b3394d62b0b5dbf4fa837fe"} Jun 02 19:04:18.787765 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.787309 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z4qbq" event={"ID":"3e9cc9fb-40ad-4f89-923b-eaa811f2a424","Type":"ContainerStarted","Data":"b7738ae1be5a0d30175cf5f6492f253f64b80cad07289dda363c2142c030d32e"} Jun 02 19:04:18.791859 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.788937 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wrs9f" event={"ID":"9efd8fa3-600f-42a8-83f5-25d80caacdd7","Type":"ContainerStarted","Data":"3a320ef04299a4d25acc68e22e036467d889cc576907d4bf6728da6439f48a41"} Jun 02 19:04:18.791859 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.791663 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zvtrb" event={"ID":"56b68433-83da-422d-be07-0a4e5c2e1f40","Type":"ContainerStarted","Data":"bd810b31772ca8c28dfd09aaee71f03fe81d3353b28be703e700b025811cb76c"} Jun 02 19:04:18.793804 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.793779 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" event={"ID":"e61262b3-9ef8-43ec-b211-15c001166361","Type":"ContainerStarted","Data":"8f7d4dde2a2c959e655ac25fbac4b377c322fa583e49f670add3bd6bb5de88f3"} Jun 02 19:04:18.795287 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.795263 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dnhhf" event={"ID":"6f947012-134a-4905-8242-57a12df7ec9f","Type":"ContainerStarted","Data":"b9a45d66885e60dd1ccd5a1b3e3be027f4452a7a787c35a8d2471d551e5f718c"} Jun 02 19:04:18.800753 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.800719 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ssh8h" podStartSLOduration=3.885110826 podStartE2EDuration="21.800708914s" podCreationTimestamp="2026-06-02 19:03:57 +0000 UTC" firstStartedPulling="2026-06-02 19:04:00.156508208 +0000 UTC m=+2.974477911" lastFinishedPulling="2026-06-02 19:04:18.072106283 +0000 UTC m=+20.890075999" observedRunningTime="2026-06-02 19:04:18.800525626 +0000 UTC m=+21.618495353" watchObservedRunningTime="2026-06-02 19:04:18.800708914 +0000 UTC m=+21.618678638" Jun 02 19:04:18.801089 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.801065 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-102.ec2.internal" podStartSLOduration=19.801060669 podStartE2EDuration="19.801060669s" podCreationTimestamp="2026-06-02 19:03:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:04:01.772831443 +0000 UTC m=+4.590801169" watchObservedRunningTime="2026-06-02 19:04:18.801060669 +0000 UTC m=+21.619030392" Jun 02 19:04:18.815359 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.815320 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wrs9f" podStartSLOduration=3.8960243820000002 podStartE2EDuration="21.815307547s" podCreationTimestamp="2026-06-02 19:03:57 +0000 UTC" firstStartedPulling="2026-06-02 19:04:00.152478278 +0000 UTC m=+2.970447981" lastFinishedPulling="2026-06-02 19:04:18.071761445 +0000 UTC m=+20.889731146" observedRunningTime="2026-06-02 19:04:18.815258201 +0000 UTC m=+21.633227925" watchObservedRunningTime="2026-06-02 19:04:18.815307547 +0000 UTC m=+21.633277273" Jun 02 19:04:18.850635 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.850599 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-zvtrb" podStartSLOduration=3.859293004 podStartE2EDuration="21.850589146s" podCreationTimestamp="2026-06-02 19:03:57 +0000 UTC" firstStartedPulling="2026-06-02 19:04:00.16087453 +0000 UTC m=+2.978844233" lastFinishedPulling="2026-06-02 19:04:18.152170671 +0000 UTC m=+20.970140375" observedRunningTime="2026-06-02 19:04:18.850229275 +0000 UTC m=+21.668199001" watchObservedRunningTime="2026-06-02 19:04:18.850589146 +0000 UTC m=+21.668558890" Jun 02 19:04:18.868100 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.868066 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-dnhhf" podStartSLOduration=4.323274465 podStartE2EDuration="21.868056707s" podCreationTimestamp="2026-06-02 19:03:57 +0000 UTC" firstStartedPulling="2026-06-02 19:04:00.151352658 +0000 UTC m=+2.969322360" lastFinishedPulling="2026-06-02 19:04:17.696134887 +0000 UTC m=+20.514104602" observedRunningTime="2026-06-02 19:04:18.867632546 +0000 UTC m=+21.685602270" watchObservedRunningTime="2026-06-02 19:04:18.868056707 +0000 UTC m=+21.686026430" Jun 02 19:04:18.881050 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:18.881012 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z4qbq" podStartSLOduration=12.079139316 podStartE2EDuration="21.880998585s" podCreationTimestamp="2026-06-02 19:03:57 +0000 UTC" firstStartedPulling="2026-06-02 19:04:00.155426496 +0000 UTC m=+2.973396205" lastFinishedPulling="2026-06-02 19:04:09.957285771 +0000 UTC m=+12.775255474" observedRunningTime="2026-06-02 19:04:18.880721399 +0000 UTC m=+21.698691124" watchObservedRunningTime="2026-06-02 19:04:18.880998585 +0000 UTC m=+21.698968310" Jun 02 19:04:19.679149 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:19.679120 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:19.679305 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:19.679122 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:19.679305 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:19.679237 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:19.679408 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:19.679347 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:19.698617 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:19.698588 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 19:04:19.798960 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:19.798926 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vd7vd" event={"ID":"ade5e001-4217-4c82-8105-53057c166c59","Type":"ContainerStarted","Data":"c52d934f5f15b77420bfb4de9a1699e761ca1d1defd0ceefdba8efbe45178436"} Jun 02 19:04:19.800629 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:19.800602 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" event={"ID":"e61262b3-9ef8-43ec-b211-15c001166361","Type":"ContainerStarted","Data":"18057b427bd1d6bfbfcf339c11f22df36eb8ceac667b4635e2d1f6f245853c99"} Jun 02 19:04:19.814380 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:19.814341 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vd7vd" podStartSLOduration=4.899520929 podStartE2EDuration="22.81432791s" podCreationTimestamp="2026-06-02 19:03:57 +0000 UTC" firstStartedPulling="2026-06-02 19:04:00.157154538 +0000 UTC m=+2.975124245" lastFinishedPulling="2026-06-02 19:04:18.071961511 +0000 UTC m=+20.889931226" observedRunningTime="2026-06-02 19:04:19.813785125 +0000 UTC m=+22.631754849" watchObservedRunningTime="2026-06-02 19:04:19.81432791 +0000 UTC m=+22.632297637" Jun 02 19:04:20.152790 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:20.152750 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:04:20.153535 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:20.153515 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:04:20.615296 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:20.615211 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:04:20.615835 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:20.615802 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z4qbq" Jun 02 19:04:20.628272 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:20.628174 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T19:04:19.698609323Z","UUID":"7025a31d-de04-4b79-8a56-f479a4b436eb","Handler":null,"Name":"","Endpoint":""} Jun 02 19:04:20.629874 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:20.629846 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 19:04:20.629874 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:20.629872 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 19:04:20.806030 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:20.806001 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" event={"ID":"e61262b3-9ef8-43ec-b211-15c001166361","Type":"ContainerStarted","Data":"d8aaf330cc74e2e815eb3ae04bca32230f0745a552d1acab9663cd45ab57ecb7"} Jun 02 19:04:20.822188 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:20.821615 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnm2n" podStartSLOduration=2.343510443 podStartE2EDuration="22.821597697s" podCreationTimestamp="2026-06-02 19:03:58 +0000 UTC" firstStartedPulling="2026-06-02 19:04:00.151382335 +0000 UTC m=+2.969352039" lastFinishedPulling="2026-06-02 19:04:20.629469585 +0000 UTC m=+23.447439293" observedRunningTime="2026-06-02 19:04:20.821518708 +0000 UTC m=+23.639488435" watchObservedRunningTime="2026-06-02 19:04:20.821597697 +0000 UTC m=+23.639567421" Jun 02 19:04:21.678863 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:21.678833 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:21.679028 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:21.678833 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:21.679028 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:21.678962 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:21.679028 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:21.679009 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:21.810738 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:21.810701 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" event={"ID":"12107e6a-8aed-4528-a741-e28f41659026","Type":"ContainerStarted","Data":"aac9f0f49b34c8de9bd6f941d3bc5e1e0bf8b8f0d0df880f0777d47fb4e50c56"} Jun 02 19:04:23.679420 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:23.679247 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:23.680160 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:23.679258 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:23.680160 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:23.679522 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:23.680160 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:23.679602 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:23.817617 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:23.817582 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" event={"ID":"12107e6a-8aed-4528-a741-e28f41659026","Type":"ContainerStarted","Data":"1a794f6ecc8032e74a7ed5f886be8aad8451b2c2454f1a33727c3d84242a3ca7"} Jun 02 19:04:23.817970 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:23.817939 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:04:23.817970 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:23.817971 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:04:23.818120 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:23.817986 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:04:23.819869 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:23.819847 2574 generic.go:358] "Generic (PLEG): container finished" podID="ee8b07c2-5feb-4d66-87aa-aee5c65834dd" containerID="d415a8c5286351509d310cc26ac45f4a8a5ca34a965748b4fca31a3d80b7eb3e" exitCode=0 Jun 02 19:04:23.819973 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:23.819893 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2q4" event={"ID":"ee8b07c2-5feb-4d66-87aa-aee5c65834dd","Type":"ContainerDied","Data":"d415a8c5286351509d310cc26ac45f4a8a5ca34a965748b4fca31a3d80b7eb3e"} Jun 02 19:04:23.836505 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:23.836483 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:04:23.836646 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:23.836632 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:04:23.842008 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:23.841968 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" podStartSLOduration=7.88505027 podStartE2EDuration="25.841956768s" podCreationTimestamp="2026-06-02 19:03:58 +0000 UTC" firstStartedPulling="2026-06-02 19:04:00.163892165 +0000 UTC m=+2.981861874" lastFinishedPulling="2026-06-02 19:04:18.120798656 +0000 UTC m=+20.938768372" observedRunningTime="2026-06-02 19:04:23.840773089 +0000 UTC m=+26.658742812" watchObservedRunningTime="2026-06-02 19:04:23.841956768 +0000 UTC m=+26.659926492" Jun 02 19:04:25.679147 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:25.678947 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:25.679557 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:25.678954 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:25.679557 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:25.679239 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:25.679557 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:25.679390 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:25.825638 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:25.825604 2574 generic.go:358] "Generic (PLEG): container finished" podID="ee8b07c2-5feb-4d66-87aa-aee5c65834dd" containerID="ea90ac58023b1731818819e0ffbc6eb3379aab336998590a454a527615f20fbe" exitCode=0 Jun 02 19:04:25.825792 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:25.825666 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2q4" event={"ID":"ee8b07c2-5feb-4d66-87aa-aee5c65834dd","Type":"ContainerDied","Data":"ea90ac58023b1731818819e0ffbc6eb3379aab336998590a454a527615f20fbe"} Jun 02 19:04:27.679871 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:27.679840 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:27.680296 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:27.679931 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:27.680296 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:27.680019 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:27.680296 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:27.680133 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:27.830549 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:27.830518 2574 generic.go:358] "Generic (PLEG): container finished" podID="ee8b07c2-5feb-4d66-87aa-aee5c65834dd" containerID="361b710d3e434a65cc3745773cf3e932f9cf04c3124ebddc6794beec3c1f0156" exitCode=0 Jun 02 19:04:27.830549 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:27.830549 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2q4" event={"ID":"ee8b07c2-5feb-4d66-87aa-aee5c65834dd","Type":"ContainerDied","Data":"361b710d3e434a65cc3745773cf3e932f9cf04c3124ebddc6794beec3c1f0156"} Jun 02 19:04:29.126205 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.126157 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-b95tp"] Jun 02 19:04:29.147365 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.147338 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:29.147512 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:29.147431 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b95tp" podUID="c641d72d-ffca-480e-9fa8-4b7200759721" Jun 02 19:04:29.289408 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.289373 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c641d72d-ffca-480e-9fa8-4b7200759721-kubelet-config\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:29.289574 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.289429 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:29.289574 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.289482 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c641d72d-ffca-480e-9fa8-4b7200759721-dbus\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:29.389832 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.389756 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:29.389972 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.389833 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c641d72d-ffca-480e-9fa8-4b7200759721-dbus\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:29.389972 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.389855 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c641d72d-ffca-480e-9fa8-4b7200759721-kubelet-config\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:29.389972 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:29.389897 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:04:29.389972 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:29.389968 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret podName:c641d72d-ffca-480e-9fa8-4b7200759721 nodeName:}" failed. No retries permitted until 2026-06-02 19:04:29.889948208 +0000 UTC m=+32.707917924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret") pod "global-pull-secret-syncer-b95tp" (UID: "c641d72d-ffca-480e-9fa8-4b7200759721") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:04:29.390126 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.389984 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c641d72d-ffca-480e-9fa8-4b7200759721-kubelet-config\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:29.390157 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.390121 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c641d72d-ffca-480e-9fa8-4b7200759721-dbus\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:29.679570 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.679533 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:29.679702 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:29.679645 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:29.679769 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.679699 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:29.679821 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:29.679789 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:29.867626 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.867594 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b95tp"] Jun 02 19:04:29.867743 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.867732 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:29.867866 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:29.867847 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b95tp" podUID="c641d72d-ffca-480e-9fa8-4b7200759721" Jun 02 19:04:29.872458 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.872435 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mp2fh"] Jun 02 19:04:29.872564 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.872548 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:29.872677 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:29.872654 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:29.873791 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.873767 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xc29k"] Jun 02 19:04:29.873877 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.873856 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:29.873938 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:29.873919 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:29.893442 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:29.893422 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:29.893535 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:29.893524 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:04:29.893583 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:29.893564 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret podName:c641d72d-ffca-480e-9fa8-4b7200759721 nodeName:}" failed. No retries permitted until 2026-06-02 19:04:30.893552149 +0000 UTC m=+33.711521852 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret") pod "global-pull-secret-syncer-b95tp" (UID: "c641d72d-ffca-480e-9fa8-4b7200759721") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:04:30.900285 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:30.900255 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:30.900722 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:30.900411 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:04:30.900722 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:30.900479 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret podName:c641d72d-ffca-480e-9fa8-4b7200759721 nodeName:}" failed. No retries permitted until 2026-06-02 19:04:32.900458991 +0000 UTC m=+35.718428700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret") pod "global-pull-secret-syncer-b95tp" (UID: "c641d72d-ffca-480e-9fa8-4b7200759721") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:04:31.303294 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:31.303270 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:31.303446 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:31.303383 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:31.303446 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:31.303439 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs podName:9e006346-8709-4a02-bf44-d9e5a59bb74b nodeName:}" failed. No retries permitted until 2026-06-02 19:05:03.303418945 +0000 UTC m=+66.121388647 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs") pod "network-metrics-daemon-mp2fh" (UID: "9e006346-8709-4a02-bf44-d9e5a59bb74b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 19:04:31.404140 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:31.404116 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nhn\" (UniqueName: \"kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn\") pod \"network-check-target-xc29k\" (UID: \"ebe8479b-96cc-4fd7-928d-fd02f2345f2a\") " pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:31.404314 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:31.404295 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 19:04:31.404384 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:31.404320 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 19:04:31.404384 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:31.404332 2574 projected.go:194] Error preparing data for projected volume kube-api-access-k6nhn for pod openshift-network-diagnostics/network-check-target-xc29k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:31.404488 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:31.404389 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn podName:ebe8479b-96cc-4fd7-928d-fd02f2345f2a nodeName:}" failed. No retries permitted until 2026-06-02 19:05:03.404370971 +0000 UTC m=+66.222340694 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6nhn" (UniqueName: "kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn") pod "network-check-target-xc29k" (UID: "ebe8479b-96cc-4fd7-928d-fd02f2345f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 19:04:31.679349 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:31.679314 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:31.679476 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:31.679383 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:31.679515 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:31.679480 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b95tp" podUID="c641d72d-ffca-480e-9fa8-4b7200759721" Jun 02 19:04:31.679515 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:31.679501 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:31.679592 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:31.679543 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:31.679668 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:31.679649 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:32.914401 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:32.914373 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:32.914759 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:32.914484 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 19:04:32.914759 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:32.914529 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret podName:c641d72d-ffca-480e-9fa8-4b7200759721 nodeName:}" failed. No retries permitted until 2026-06-02 19:04:36.91451605 +0000 UTC m=+39.732485752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret") pod "global-pull-secret-syncer-b95tp" (UID: "c641d72d-ffca-480e-9fa8-4b7200759721") : object "kube-system"/"original-pull-secret" not registered Jun 02 19:04:33.678844 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:33.678811 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:33.678961 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:33.678846 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:33.678961 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:33.678811 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:33.678961 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:33.678943 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b95tp" podUID="c641d72d-ffca-480e-9fa8-4b7200759721" Jun 02 19:04:33.679119 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:33.679053 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc29k" podUID="ebe8479b-96cc-4fd7-928d-fd02f2345f2a" Jun 02 19:04:33.679171 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:33.679139 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp2fh" podUID="9e006346-8709-4a02-bf44-d9e5a59bb74b" Jun 02 19:04:34.032038 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.031966 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-102.ec2.internal" event="NodeReady" Jun 02 19:04:34.032502 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.032109 2574 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 19:04:34.065767 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.065700 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f7fdc8998-tqs6k"] Jun 02 19:04:34.078032 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.078005 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.079122 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.078861 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2lskv"] Jun 02 19:04:34.080369 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.080343 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 19:04:34.080481 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.080384 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 19:04:34.080656 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.080634 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 19:04:34.080911 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.080839 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fds9f\"" Jun 02 19:04:34.086074 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.086056 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 19:04:34.090912 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.090870 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f7fdc8998-tqs6k"] Jun 02 19:04:34.090912 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.090899 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mv67v"] Jun 02 19:04:34.091057 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.091040 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.095492 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.095474 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 19:04:34.095796 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.095776 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 19:04:34.096179 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.096153 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 19:04:34.096287 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.096153 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jcshd\"" Jun 02 19:04:34.097412 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.096828 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 19:04:34.101997 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.101977 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mv67v"] Jun 02 19:04:34.102097 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.102005 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-26xk6"] Jun 02 19:04:34.102159 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.102145 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mv67v" Jun 02 19:04:34.104808 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.104606 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 19:04:34.104808 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.104647 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 19:04:34.104808 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.104654 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z6cjs\"" Jun 02 19:04:34.104808 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.104606 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 19:04:34.113409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.113389 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2lskv"] Jun 02 19:04:34.113492 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.113416 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-26xk6"] Jun 02 19:04:34.113543 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.113517 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.115961 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.115942 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t48t7\"" Jun 02 19:04:34.116058 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.115969 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 19:04:34.116058 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.116025 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 19:04:34.225190 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.225158 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-ca-trust-extracted\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.225190 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.225200 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-image-registry-private-configuration\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.225465 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.225244 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-installation-pull-secrets\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.225465 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.225284 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-bound-sa-token\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.225465 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.225307 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/456f7870-5977-4ee2-8d13-2ae8a34f3119-data-volume\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.225465 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.225334 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5494ce57-6970-4bd2-aae6-6c83780cbc49-tmp-dir\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.225854 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.225811 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4d8l\" (UniqueName: \"kubernetes.io/projected/456f7870-5977-4ee2-8d13-2ae8a34f3119-kube-api-access-b4d8l\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.226005 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.225880 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5494ce57-6970-4bd2-aae6-6c83780cbc49-config-volume\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.226005 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.225939 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-registry-certificates\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.226005 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.225981 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/456f7870-5977-4ee2-8d13-2ae8a34f3119-crio-socket\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.226116 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.226015 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkplz\" (UniqueName: \"kubernetes.io/projected/5494ce57-6970-4bd2-aae6-6c83780cbc49-kube-api-access-pkplz\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.226116 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.226049 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8brl\" (UniqueName: \"kubernetes.io/projected/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-kube-api-access-w8brl\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.226116 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.226080 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/456f7870-5977-4ee2-8d13-2ae8a34f3119-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.226116 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.226108 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5494ce57-6970-4bd2-aae6-6c83780cbc49-metrics-tls\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.226321 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.226133 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-registry-tls\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.226821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.226624 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-trusted-ca\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.226821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.226705 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/456f7870-5977-4ee2-8d13-2ae8a34f3119-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.226821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.226749 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af-cert\") pod \"ingress-canary-mv67v\" (UID: \"5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af\") " pod="openshift-ingress-canary/ingress-canary-mv67v" Jun 02 19:04:34.226821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.226813 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs9s2\" (UniqueName: \"kubernetes.io/projected/5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af-kube-api-access-fs9s2\") pod \"ingress-canary-mv67v\" (UID: \"5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af\") " pod="openshift-ingress-canary/ingress-canary-mv67v" Jun 02 19:04:34.328234 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.328139 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/456f7870-5977-4ee2-8d13-2ae8a34f3119-crio-socket\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.328234 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.328187 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pkplz\" (UniqueName: \"kubernetes.io/projected/5494ce57-6970-4bd2-aae6-6c83780cbc49-kube-api-access-pkplz\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.328234 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.328211 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w8brl\" (UniqueName: \"kubernetes.io/projected/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-kube-api-access-w8brl\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.328482 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.328251 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/456f7870-5977-4ee2-8d13-2ae8a34f3119-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.328482 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.328276 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5494ce57-6970-4bd2-aae6-6c83780cbc49-metrics-tls\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.328482 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.328387 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/456f7870-5977-4ee2-8d13-2ae8a34f3119-crio-socket\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.328482 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.328463 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-registry-tls\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.328685 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.328500 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-trusted-ca\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.328685 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.328535 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/456f7870-5977-4ee2-8d13-2ae8a34f3119-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.328685 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.328564 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af-cert\") pod \"ingress-canary-mv67v\" (UID: \"5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af\") " pod="openshift-ingress-canary/ingress-canary-mv67v" Jun 02 19:04:34.328685 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.328594 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fs9s2\" (UniqueName: \"kubernetes.io/projected/5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af-kube-api-access-fs9s2\") pod \"ingress-canary-mv67v\" (UID: \"5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af\") " pod="openshift-ingress-canary/ingress-canary-mv67v" Jun 02 19:04:34.329177 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329128 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-ca-trust-extracted\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.329177 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329175 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-image-registry-private-configuration\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.329385 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329210 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-installation-pull-secrets\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.329385 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329256 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-bound-sa-token\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.329385 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329282 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/456f7870-5977-4ee2-8d13-2ae8a34f3119-data-volume\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.329385 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329317 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5494ce57-6970-4bd2-aae6-6c83780cbc49-tmp-dir\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.329385 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329329 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/456f7870-5977-4ee2-8d13-2ae8a34f3119-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.329608 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329577 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/456f7870-5977-4ee2-8d13-2ae8a34f3119-data-volume\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.329653 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329628 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b4d8l\" (UniqueName: \"kubernetes.io/projected/456f7870-5977-4ee2-8d13-2ae8a34f3119-kube-api-access-b4d8l\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.329696 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329651 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-trusted-ca\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.329696 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329667 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5494ce57-6970-4bd2-aae6-6c83780cbc49-config-volume\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.329788 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.329704 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-registry-certificates\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.330202 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.330159 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5494ce57-6970-4bd2-aae6-6c83780cbc49-config-volume\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.330334 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.330205 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5494ce57-6970-4bd2-aae6-6c83780cbc49-tmp-dir\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.330513 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.330490 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-registry-certificates\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.330770 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.330749 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-ca-trust-extracted\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.333449 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.333425 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-image-registry-private-configuration\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.333725 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.333679 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/456f7870-5977-4ee2-8d13-2ae8a34f3119-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.333725 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.333718 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-installation-pull-secrets\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.333902 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.333730 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5494ce57-6970-4bd2-aae6-6c83780cbc49-metrics-tls\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.333902 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.333812 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-registry-tls\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.333902 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.333881 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af-cert\") pod \"ingress-canary-mv67v\" (UID: \"5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af\") " pod="openshift-ingress-canary/ingress-canary-mv67v" Jun 02 19:04:34.337459 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.337406 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-bound-sa-token\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.337640 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.337615 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkplz\" (UniqueName: \"kubernetes.io/projected/5494ce57-6970-4bd2-aae6-6c83780cbc49-kube-api-access-pkplz\") pod \"dns-default-26xk6\" (UID: \"5494ce57-6970-4bd2-aae6-6c83780cbc49\") " pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:34.337739 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.337654 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs9s2\" (UniqueName: \"kubernetes.io/projected/5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af-kube-api-access-fs9s2\") pod \"ingress-canary-mv67v\" (UID: \"5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af\") " pod="openshift-ingress-canary/ingress-canary-mv67v" Jun 02 19:04:34.338052 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.338006 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8brl\" (UniqueName: \"kubernetes.io/projected/624f5b44-0801-4a57-bb6b-f22c6bcd2afd-kube-api-access-w8brl\") pod \"image-registry-f7fdc8998-tqs6k\" (UID: \"624f5b44-0801-4a57-bb6b-f22c6bcd2afd\") " pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.339551 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.339529 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4d8l\" (UniqueName: \"kubernetes.io/projected/456f7870-5977-4ee2-8d13-2ae8a34f3119-kube-api-access-b4d8l\") pod \"insights-runtime-extractor-2lskv\" (UID: \"456f7870-5977-4ee2-8d13-2ae8a34f3119\") " pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.393178 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.393157 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:34.403962 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.403944 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2lskv" Jun 02 19:04:34.412524 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.412505 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mv67v" Jun 02 19:04:34.423188 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:34.423166 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:35.679524 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:35.679487 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:35.680170 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:35.679632 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:04:35.680170 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:35.679825 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:04:35.682520 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:35.682490 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 19:04:35.682658 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:35.682618 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dks2v\"" Jun 02 19:04:35.682658 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:35.682637 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 19:04:35.682769 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:35.682666 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 19:04:35.682815 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:35.682792 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mp95k\"" Jun 02 19:04:35.682921 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:35.682890 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 19:04:36.129019 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.128989 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2lskv"] Jun 02 19:04:36.201514 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:36.201487 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod456f7870_5977_4ee2_8d13_2ae8a34f3119.slice/crio-11149552399f4ff87c5b0c1b604ac6f9a9ef75940e15dc8b337ce2c66365d4ee WatchSource:0}: Error finding container 11149552399f4ff87c5b0c1b604ac6f9a9ef75940e15dc8b337ce2c66365d4ee: Status 404 returned error can't find the container with id 11149552399f4ff87c5b0c1b604ac6f9a9ef75940e15dc8b337ce2c66365d4ee Jun 02 19:04:36.348236 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.348179 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-26xk6"] Jun 02 19:04:36.350999 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.350978 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mv67v"] Jun 02 19:04:36.351404 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:36.351382 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5494ce57_6970_4bd2_aae6_6c83780cbc49.slice/crio-c81dec8f30b3dfdd5b63820fe5f9a81772492a3652c543d3058950ef03501b56 WatchSource:0}: Error finding container c81dec8f30b3dfdd5b63820fe5f9a81772492a3652c543d3058950ef03501b56: Status 404 returned error can't find the container with id c81dec8f30b3dfdd5b63820fe5f9a81772492a3652c543d3058950ef03501b56 Jun 02 19:04:36.354587 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.354470 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f7fdc8998-tqs6k"] Jun 02 19:04:36.356662 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:36.356634 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdac944_f6ec_4d0d_8eac_31d0f8b3c5af.slice/crio-822d95c10e6947dcf0ee848dae677e219ed95d30647d90637f81cfa0de1e00a6 WatchSource:0}: Error finding container 822d95c10e6947dcf0ee848dae677e219ed95d30647d90637f81cfa0de1e00a6: Status 404 returned error can't find the container with id 822d95c10e6947dcf0ee848dae677e219ed95d30647d90637f81cfa0de1e00a6 Jun 02 19:04:36.357968 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:36.357942 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624f5b44_0801_4a57_bb6b_f22c6bcd2afd.slice/crio-a4e252e852c59b0e7cfc810d62415fd660282b6d6057fe3bc5e05a4aaf7c56bd WatchSource:0}: Error finding container a4e252e852c59b0e7cfc810d62415fd660282b6d6057fe3bc5e05a4aaf7c56bd: Status 404 returned error can't find the container with id a4e252e852c59b0e7cfc810d62415fd660282b6d6057fe3bc5e05a4aaf7c56bd Jun 02 19:04:36.848203 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.848106 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mv67v" event={"ID":"5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af","Type":"ContainerStarted","Data":"822d95c10e6947dcf0ee848dae677e219ed95d30647d90637f81cfa0de1e00a6"} Jun 02 19:04:36.849618 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.849589 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-26xk6" event={"ID":"5494ce57-6970-4bd2-aae6-6c83780cbc49","Type":"ContainerStarted","Data":"c81dec8f30b3dfdd5b63820fe5f9a81772492a3652c543d3058950ef03501b56"} Jun 02 19:04:36.851365 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.851339 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" event={"ID":"624f5b44-0801-4a57-bb6b-f22c6bcd2afd","Type":"ContainerStarted","Data":"02910e9621e5e7b475623555614bde2b159c6ce0e93393569686453f9818b2ca"} Jun 02 19:04:36.851477 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.851374 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" event={"ID":"624f5b44-0801-4a57-bb6b-f22c6bcd2afd","Type":"ContainerStarted","Data":"a4e252e852c59b0e7cfc810d62415fd660282b6d6057fe3bc5e05a4aaf7c56bd"} Jun 02 19:04:36.851803 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.851786 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:36.853068 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.853010 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2lskv" event={"ID":"456f7870-5977-4ee2-8d13-2ae8a34f3119","Type":"ContainerStarted","Data":"9be1cf96c0c72507f5c50497db6ec68d11ef575d96ee9c4bd82b28d8abf2f1e9"} Jun 02 19:04:36.853068 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.853050 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2lskv" event={"ID":"456f7870-5977-4ee2-8d13-2ae8a34f3119","Type":"ContainerStarted","Data":"11149552399f4ff87c5b0c1b604ac6f9a9ef75940e15dc8b337ce2c66365d4ee"} Jun 02 19:04:36.858300 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.857974 2574 generic.go:358] "Generic (PLEG): container finished" podID="ee8b07c2-5feb-4d66-87aa-aee5c65834dd" containerID="f87faf9e04f4dfca89a42372a82c32bb6eb9ce9bffeefa69141995da9b1e6675" exitCode=0 Jun 02 19:04:36.858300 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.858015 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2q4" event={"ID":"ee8b07c2-5feb-4d66-87aa-aee5c65834dd","Type":"ContainerDied","Data":"f87faf9e04f4dfca89a42372a82c32bb6eb9ce9bffeefa69141995da9b1e6675"} Jun 02 19:04:36.898682 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.898633 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" podStartSLOduration=8.898615541 podStartE2EDuration="8.898615541s" podCreationTimestamp="2026-06-02 19:04:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:04:36.873932529 +0000 UTC m=+39.691902246" watchObservedRunningTime="2026-06-02 19:04:36.898615541 +0000 UTC m=+39.716585267" Jun 02 19:04:36.947840 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.947812 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:36.950520 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:36.950500 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c641d72d-ffca-480e-9fa8-4b7200759721-original-pull-secret\") pod \"global-pull-secret-syncer-b95tp\" (UID: \"c641d72d-ffca-480e-9fa8-4b7200759721\") " pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:37.192088 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.192064 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b95tp" Jun 02 19:04:37.332437 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.332381 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b95tp"] Jun 02 19:04:37.645255 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.645184 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-t5v5v"] Jun 02 19:04:37.648236 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.648064 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.650559 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.650531 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 02 19:04:37.650675 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.650533 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 02 19:04:37.650762 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.650744 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 19:04:37.650840 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.650806 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 19:04:37.651273 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.651110 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mdd6q\"" Jun 02 19:04:37.651273 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.651130 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 19:04:37.656804 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.656767 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-t5v5v"] Jun 02 19:04:37.672985 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.672959 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7"] Jun 02 19:04:37.675153 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.675130 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.676301 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:37.676278 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc641d72d_ffca_480e_9fa8_4b7200759721.slice/crio-c9ffbcd4f5bb9b0677488de328c581699079660edefd6b2563646c3d0b77f340 WatchSource:0}: Error finding container c9ffbcd4f5bb9b0677488de328c581699079660edefd6b2563646c3d0b77f340: Status 404 returned error can't find the container with id c9ffbcd4f5bb9b0677488de328c581699079660edefd6b2563646c3d0b77f340 Jun 02 19:04:37.678491 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.678275 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 02 19:04:37.678491 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.678355 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-x75dj\"" Jun 02 19:04:37.678491 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.678385 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 02 19:04:37.679069 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.679048 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 02 19:04:37.689971 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.689943 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7"] Jun 02 19:04:37.701560 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.701537 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wkzf4"] Jun 02 19:04:37.703501 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.703483 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.705799 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.705778 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 19:04:37.706045 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.706026 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-52x9g\"" Jun 02 19:04:37.706132 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.706085 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 19:04:37.707572 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.707549 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 19:04:37.754409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.754136 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.754409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.754178 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/febbc43c-abfd-4dee-99f6-4d33cdc29c37-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.754409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.754255 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85509742-48ce-41fc-a98c-e8c35c87b828-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.754409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.754284 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.754409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.754310 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ltgp\" (UniqueName: \"kubernetes.io/projected/85509742-48ce-41fc-a98c-e8c35c87b828-kube-api-access-9ltgp\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.754409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.754336 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjlf5\" (UniqueName: \"kubernetes.io/projected/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-api-access-hjlf5\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.754752 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.754443 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/febbc43c-abfd-4dee-99f6-4d33cdc29c37-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.754752 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.754496 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85509742-48ce-41fc-a98c-e8c35c87b828-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.754752 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.754524 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/85509742-48ce-41fc-a98c-e8c35c87b828-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.754752 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.754552 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.855289 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855195 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-textfile\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.855289 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855258 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-tls\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855301 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/febbc43c-abfd-4dee-99f6-4d33cdc29c37-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855346 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85509742-48ce-41fc-a98c-e8c35c87b828-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855375 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/85509742-48ce-41fc-a98c-e8c35c87b828-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855404 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855439 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855467 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/febbc43c-abfd-4dee-99f6-4d33cdc29c37-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855501 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-metrics-client-ca\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855540 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85509742-48ce-41fc-a98c-e8c35c87b828-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855593 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855619 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-sys\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855644 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855675 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9ltgp\" (UniqueName: \"kubernetes.io/projected/85509742-48ce-41fc-a98c-e8c35c87b828-kube-api-access-9ltgp\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.855703 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855700 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hjlf5\" (UniqueName: \"kubernetes.io/projected/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-api-access-hjlf5\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.856335 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855729 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-root\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.856335 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855767 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnhtt\" (UniqueName: \"kubernetes.io/projected/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-kube-api-access-bnhtt\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.856335 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855799 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-accelerators-collector-config\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.856335 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.855827 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-wtmp\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.856335 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.856203 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/febbc43c-abfd-4dee-99f6-4d33cdc29c37-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.856580 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:37.856349 2574 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jun 02 19:04:37.856580 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:37.856414 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-tls podName:febbc43c-abfd-4dee-99f6-4d33cdc29c37 nodeName:}" failed. No retries permitted until 2026-06-02 19:04:38.356390788 +0000 UTC m=+41.174360500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-tls") pod "kube-state-metrics-8469c486bb-6jsl7" (UID: "febbc43c-abfd-4dee-99f6-4d33cdc29c37") : secret "kube-state-metrics-tls" not found Jun 02 19:04:37.856691 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:37.856678 2574 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 02 19:04:37.856741 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:37.856731 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85509742-48ce-41fc-a98c-e8c35c87b828-openshift-state-metrics-tls podName:85509742-48ce-41fc-a98c-e8c35c87b828 nodeName:}" failed. No retries permitted until 2026-06-02 19:04:38.356716459 +0000 UTC m=+41.174686176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/85509742-48ce-41fc-a98c-e8c35c87b828-openshift-state-metrics-tls") pod "openshift-state-metrics-85658796db-t5v5v" (UID: "85509742-48ce-41fc-a98c-e8c35c87b828") : secret "openshift-state-metrics-tls" not found Jun 02 19:04:37.856963 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.856817 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85509742-48ce-41fc-a98c-e8c35c87b828-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.858550 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.857663 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/febbc43c-abfd-4dee-99f6-4d33cdc29c37-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.858550 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.857814 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.859870 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.859806 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.860614 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.860594 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85509742-48ce-41fc-a98c-e8c35c87b828-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.861650 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.861616 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-26xk6" event={"ID":"5494ce57-6970-4bd2-aae6-6c83780cbc49","Type":"ContainerStarted","Data":"a135cbe4499c84c08da4c22ba7cb247be276ecc6bb91742fad548a5de86f7e44"} Jun 02 19:04:37.863031 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.863005 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b95tp" event={"ID":"c641d72d-ffca-480e-9fa8-4b7200759721","Type":"ContainerStarted","Data":"c9ffbcd4f5bb9b0677488de328c581699079660edefd6b2563646c3d0b77f340"} Jun 02 19:04:37.865405 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.865357 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2lskv" event={"ID":"456f7870-5977-4ee2-8d13-2ae8a34f3119","Type":"ContainerStarted","Data":"0a2b010f5ca9b13f8081a8840c2b7caa71de50fb86e5b520d6dce30f27ef43ac"} Jun 02 19:04:37.867050 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.866978 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjlf5\" (UniqueName: \"kubernetes.io/projected/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-api-access-hjlf5\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:37.868128 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.868105 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ltgp\" (UniqueName: \"kubernetes.io/projected/85509742-48ce-41fc-a98c-e8c35c87b828-kube-api-access-9ltgp\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:37.868751 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.868362 2574 generic.go:358] "Generic (PLEG): container finished" podID="ee8b07c2-5feb-4d66-87aa-aee5c65834dd" containerID="61cbfed5527b8da8199596789b274f22c2029df381d1539cf2fa7e56e3bd8dfb" exitCode=0 Jun 02 19:04:37.868751 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.868437 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2q4" event={"ID":"ee8b07c2-5feb-4d66-87aa-aee5c65834dd","Type":"ContainerDied","Data":"61cbfed5527b8da8199596789b274f22c2029df381d1539cf2fa7e56e3bd8dfb"} Jun 02 19:04:37.956853 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.956825 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-metrics-client-ca\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.956981 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.956903 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-sys\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.956981 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.956921 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.956981 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.956941 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-root\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.957211 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.957000 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bnhtt\" (UniqueName: \"kubernetes.io/projected/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-kube-api-access-bnhtt\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.957211 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.957043 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-accelerators-collector-config\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.957211 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.957075 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-wtmp\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.957211 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.957002 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-sys\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.957211 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.957189 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-textfile\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.957495 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.957239 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-tls\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.957495 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.957385 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-metrics-client-ca\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.957495 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.957474 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-root\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.957647 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.957527 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-wtmp\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.957821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.957798 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-accelerators-collector-config\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.957904 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.957882 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-textfile\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.959902 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.959824 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-tls\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.960309 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.960152 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:37.970680 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:37.970634 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnhtt\" (UniqueName: \"kubernetes.io/projected/9dbf7bb9-a19a-4279-ae78-0e0129e976a7-kube-api-access-bnhtt\") pod \"node-exporter-wkzf4\" (UID: \"9dbf7bb9-a19a-4279-ae78-0e0129e976a7\") " pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:38.016224 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.016200 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wkzf4" Jun 02 19:04:38.360194 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.360120 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/85509742-48ce-41fc-a98c-e8c35c87b828-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:38.360194 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.360169 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:38.363257 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.363193 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/febbc43c-abfd-4dee-99f6-4d33cdc29c37-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-6jsl7\" (UID: \"febbc43c-abfd-4dee-99f6-4d33cdc29c37\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:38.363827 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.363790 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/85509742-48ce-41fc-a98c-e8c35c87b828-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-t5v5v\" (UID: \"85509742-48ce-41fc-a98c-e8c35c87b828\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:38.470324 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:38.470282 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dbf7bb9_a19a_4279_ae78_0e0129e976a7.slice/crio-d96f2b76f1c3d867b27b05ad11d48d4e4eb8c7cdf06fc7cff45c8c143dd19cac WatchSource:0}: Error finding container d96f2b76f1c3d867b27b05ad11d48d4e4eb8c7cdf06fc7cff45c8c143dd19cac: Status 404 returned error can't find the container with id d96f2b76f1c3d867b27b05ad11d48d4e4eb8c7cdf06fc7cff45c8c143dd19cac Jun 02 19:04:38.530395 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.530372 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-6q4kq"] Jun 02 19:04:38.532267 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.532247 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-6q4kq" Jun 02 19:04:38.534594 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.534573 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 02 19:04:38.534685 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.534599 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 02 19:04:38.534897 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.534884 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kxw7r\"" Jun 02 19:04:38.545062 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.545023 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-6q4kq"] Jun 02 19:04:38.560984 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.560964 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" Jun 02 19:04:38.591713 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.588432 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" Jun 02 19:04:38.662256 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.662203 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djw4d\" (UniqueName: \"kubernetes.io/projected/e25f59c3-6117-4dbb-8dc5-dd26b49d43f8-kube-api-access-djw4d\") pod \"downloads-c65b8dc4b-6q4kq\" (UID: \"e25f59c3-6117-4dbb-8dc5-dd26b49d43f8\") " pod="openshift-console/downloads-c65b8dc4b-6q4kq" Jun 02 19:04:38.763517 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.763481 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-djw4d\" (UniqueName: \"kubernetes.io/projected/e25f59c3-6117-4dbb-8dc5-dd26b49d43f8-kube-api-access-djw4d\") pod \"downloads-c65b8dc4b-6q4kq\" (UID: \"e25f59c3-6117-4dbb-8dc5-dd26b49d43f8\") " pod="openshift-console/downloads-c65b8dc4b-6q4kq" Jun 02 19:04:38.772227 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.772194 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djw4d\" (UniqueName: \"kubernetes.io/projected/e25f59c3-6117-4dbb-8dc5-dd26b49d43f8-kube-api-access-djw4d\") pod \"downloads-c65b8dc4b-6q4kq\" (UID: \"e25f59c3-6117-4dbb-8dc5-dd26b49d43f8\") " pod="openshift-console/downloads-c65b8dc4b-6q4kq" Jun 02 19:04:38.798744 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.798522 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:04:38.801516 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.801493 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.804194 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.804003 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bqqz8\"" Jun 02 19:04:38.804194 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.804025 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 02 19:04:38.804346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.804272 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 02 19:04:38.804346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.804331 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 02 19:04:38.804602 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.804580 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 02 19:04:38.804602 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.804588 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 02 19:04:38.804966 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.804767 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 02 19:04:38.804966 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.804840 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 02 19:04:38.805146 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.805029 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 02 19:04:38.805146 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.805135 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 02 19:04:38.816028 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.815986 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:04:38.841567 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.841525 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-6q4kq" Jun 02 19:04:38.874017 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.873984 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-26xk6" event={"ID":"5494ce57-6970-4bd2-aae6-6c83780cbc49","Type":"ContainerStarted","Data":"cdbe4d4e6a18bf037aac36e5ea9aacf83e21766233934759546f967104fb3725"} Jun 02 19:04:38.874450 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.874209 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:38.878689 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.878646 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw2q4" event={"ID":"ee8b07c2-5feb-4d66-87aa-aee5c65834dd","Type":"ContainerStarted","Data":"211073d75a6b3a43fe6127c9178cd11738c5d254ade095f75c3222d7179b952e"} Jun 02 19:04:38.879760 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.879732 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wkzf4" event={"ID":"9dbf7bb9-a19a-4279-ae78-0e0129e976a7","Type":"ContainerStarted","Data":"d96f2b76f1c3d867b27b05ad11d48d4e4eb8c7cdf06fc7cff45c8c143dd19cac"} Jun 02 19:04:38.881388 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.881240 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mv67v" event={"ID":"5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af","Type":"ContainerStarted","Data":"9016d4bb69789a01af1ce137a2a6792453090861ff8ba94c82931b7df7ef4648"} Jun 02 19:04:38.891338 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.891263 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-26xk6" podStartSLOduration=3.542124283 podStartE2EDuration="4.891249174s" podCreationTimestamp="2026-06-02 19:04:34 +0000 UTC" firstStartedPulling="2026-06-02 19:04:36.355648252 +0000 UTC m=+39.173617958" lastFinishedPulling="2026-06-02 19:04:37.704773134 +0000 UTC m=+40.522742849" observedRunningTime="2026-06-02 19:04:38.889896952 +0000 UTC m=+41.707866688" watchObservedRunningTime="2026-06-02 19:04:38.891249174 +0000 UTC m=+41.709218901" Jun 02 19:04:38.918184 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.918098 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mv67v" podStartSLOduration=2.7847580020000002 podStartE2EDuration="4.91808385s" podCreationTimestamp="2026-06-02 19:04:34 +0000 UTC" firstStartedPulling="2026-06-02 19:04:36.358757811 +0000 UTC m=+39.176727527" lastFinishedPulling="2026-06-02 19:04:38.49208367 +0000 UTC m=+41.310053375" observedRunningTime="2026-06-02 19:04:38.917413889 +0000 UTC m=+41.735383620" watchObservedRunningTime="2026-06-02 19:04:38.91808385 +0000 UTC m=+41.736053575" Jun 02 19:04:38.944853 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.944661 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vw2q4" podStartSLOduration=5.858018639 podStartE2EDuration="41.944644943s" podCreationTimestamp="2026-06-02 19:03:57 +0000 UTC" firstStartedPulling="2026-06-02 19:04:00.158502473 +0000 UTC m=+2.976472176" lastFinishedPulling="2026-06-02 19:04:36.245128778 +0000 UTC m=+39.063098480" observedRunningTime="2026-06-02 19:04:38.944409987 +0000 UTC m=+41.762379736" watchObservedRunningTime="2026-06-02 19:04:38.944644943 +0000 UTC m=+41.762614671" Jun 02 19:04:38.965766 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.965733 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-volume\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966196 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.965884 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966196 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.965925 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966196 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.966003 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966196 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.966032 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966196 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.966106 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966196 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.966154 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-web-config\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966196 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.966179 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966597 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.966229 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-out\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966597 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.966258 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966597 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.966295 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt68m\" (UniqueName: \"kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-kube-api-access-gt68m\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966597 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.966369 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:38.966597 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:38.966507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.067805 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.067775 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gt68m\" (UniqueName: \"kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-kube-api-access-gt68m\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.067973 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.067830 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.067973 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.067861 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.067973 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.067891 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-volume\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.067973 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.067950 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.068183 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.067974 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.068183 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.068016 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.068183 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.068043 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.068183 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.068092 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.068183 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.068135 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-web-config\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.069467 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.068157 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.069467 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.068517 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.069467 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.068542 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-out\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.069467 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.068581 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.069467 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:39.068964 2574 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jun 02 19:04:39.069467 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:39.069038 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-main-tls podName:f0d5f2e4-5750-4463-b72f-dd93bb136eee nodeName:}" failed. No retries permitted until 2026-06-02 19:04:39.569014014 +0000 UTC m=+42.386983731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee") : secret "alertmanager-main-tls" not found Jun 02 19:04:39.070598 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.069909 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.070598 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.070096 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.071644 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.071601 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.072670 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.072647 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.073832 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.073683 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-out\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.073832 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.073773 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-volume\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.073986 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.073847 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.074162 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.074105 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.074333 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.074314 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-web-config\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.074899 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.074881 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.079572 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.079548 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt68m\" (UniqueName: \"kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-kube-api-access-gt68m\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.291490 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.290843 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-t5v5v"] Jun 02 19:04:39.293389 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:39.293356 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85509742_48ce_41fc_a98c_e8c35c87b828.slice/crio-2eda07abbda47e034f8497a53140ebf261a7c4af5da178195bbe8c1299182448 WatchSource:0}: Error finding container 2eda07abbda47e034f8497a53140ebf261a7c4af5da178195bbe8c1299182448: Status 404 returned error can't find the container with id 2eda07abbda47e034f8497a53140ebf261a7c4af5da178195bbe8c1299182448 Jun 02 19:04:39.521074 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.521047 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-6q4kq"] Jun 02 19:04:39.523509 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:39.523479 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode25f59c3_6117_4dbb_8dc5_dd26b49d43f8.slice/crio-f5385446975748ef25e6e68644e7746f730396251d6f8daade56b8782c70c879 WatchSource:0}: Error finding container f5385446975748ef25e6e68644e7746f730396251d6f8daade56b8782c70c879: Status 404 returned error can't find the container with id f5385446975748ef25e6e68644e7746f730396251d6f8daade56b8782c70c879 Jun 02 19:04:39.527077 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.527056 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7"] Jun 02 19:04:39.572864 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.572829 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.575643 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.575618 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.713305 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.713236 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:04:39.734203 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:39.734171 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfebbc43c_abfd_4dee_99f6_4d33cdc29c37.slice/crio-873735da706cbbcd2d578075229aa9e95e7a325ce26099495e5c0c08cf4ea1f9 WatchSource:0}: Error finding container 873735da706cbbcd2d578075229aa9e95e7a325ce26099495e5c0c08cf4ea1f9: Status 404 returned error can't find the container with id 873735da706cbbcd2d578075229aa9e95e7a325ce26099495e5c0c08cf4ea1f9 Jun 02 19:04:39.884319 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.884272 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:04:39.886087 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.886038 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wkzf4" event={"ID":"9dbf7bb9-a19a-4279-ae78-0e0129e976a7","Type":"ContainerStarted","Data":"56b4a746ad2a3096c78737dcaafe805b48d4968198720868fdb08a26b1c3ac2f"} Jun 02 19:04:39.887412 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.887370 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" event={"ID":"febbc43c-abfd-4dee-99f6-4d33cdc29c37","Type":"ContainerStarted","Data":"873735da706cbbcd2d578075229aa9e95e7a325ce26099495e5c0c08cf4ea1f9"} Jun 02 19:04:39.888632 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:39.888607 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d5f2e4_5750_4463_b72f_dd93bb136eee.slice/crio-2b03f87af10ff74ee8fb31a6d0048ad72edfaaa1bc47f0bdcd0903f98f694a2f WatchSource:0}: Error finding container 2b03f87af10ff74ee8fb31a6d0048ad72edfaaa1bc47f0bdcd0903f98f694a2f: Status 404 returned error can't find the container with id 2b03f87af10ff74ee8fb31a6d0048ad72edfaaa1bc47f0bdcd0903f98f694a2f Jun 02 19:04:39.889871 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.889848 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" event={"ID":"85509742-48ce-41fc-a98c-e8c35c87b828","Type":"ContainerStarted","Data":"cde8562d014b8adcabf64ab6fdf1fd930f10b27dd28fb8a869c4e9e84c92cc64"} Jun 02 19:04:39.889944 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.889879 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" event={"ID":"85509742-48ce-41fc-a98c-e8c35c87b828","Type":"ContainerStarted","Data":"64af18a0186dd8914aa17618b33e51f3fd4fa0b3fbd598ccd760d1839cc38877"} Jun 02 19:04:39.889944 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.889893 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" event={"ID":"85509742-48ce-41fc-a98c-e8c35c87b828","Type":"ContainerStarted","Data":"2eda07abbda47e034f8497a53140ebf261a7c4af5da178195bbe8c1299182448"} Jun 02 19:04:39.891473 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.891450 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-6q4kq" event={"ID":"e25f59c3-6117-4dbb-8dc5-dd26b49d43f8","Type":"ContainerStarted","Data":"f5385446975748ef25e6e68644e7746f730396251d6f8daade56b8782c70c879"} Jun 02 19:04:39.894133 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:39.894042 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2lskv" event={"ID":"456f7870-5977-4ee2-8d13-2ae8a34f3119","Type":"ContainerStarted","Data":"26b4156cae1b35ab754580818dc9372a7856d1d8028a0a900a43c59e587a2d89"} Jun 02 19:04:40.717924 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.716747 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2lskv" podStartSLOduration=3.899149156 podStartE2EDuration="6.71672903s" podCreationTimestamp="2026-06-02 19:04:34 +0000 UTC" firstStartedPulling="2026-06-02 19:04:36.344319289 +0000 UTC m=+39.162288991" lastFinishedPulling="2026-06-02 19:04:39.161899157 +0000 UTC m=+41.979868865" observedRunningTime="2026-06-02 19:04:39.92569092 +0000 UTC m=+42.743660644" watchObservedRunningTime="2026-06-02 19:04:40.71672903 +0000 UTC m=+43.534698755" Jun 02 19:04:40.717924 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.716941 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-689cf468df-sh2kv"] Jun 02 19:04:40.720864 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.719853 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.723436 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.723406 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 02 19:04:40.723691 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.723674 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 02 19:04:40.723892 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.723876 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-23rg6s5agi594\"" Jun 02 19:04:40.724083 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.724067 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 02 19:04:40.727324 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.724290 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 02 19:04:40.727324 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.725359 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-v6c64\"" Jun 02 19:04:40.727324 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.725600 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 02 19:04:40.734364 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.734323 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-689cf468df-sh2kv"] Jun 02 19:04:40.883316 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.882872 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.883316 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.882918 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/328db247-e97e-4842-85b2-073684347027-metrics-client-ca\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.883316 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.882962 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.883316 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.883012 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.883316 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.883072 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-tls\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.883316 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.883098 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.883316 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.883125 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-grpc-tls\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.883316 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.883153 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpx4l\" (UniqueName: \"kubernetes.io/projected/328db247-e97e-4842-85b2-073684347027-kube-api-access-dpx4l\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.900365 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.900332 2574 generic.go:358] "Generic (PLEG): container finished" podID="9dbf7bb9-a19a-4279-ae78-0e0129e976a7" containerID="56b4a746ad2a3096c78737dcaafe805b48d4968198720868fdb08a26b1c3ac2f" exitCode=0 Jun 02 19:04:40.900754 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.900431 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wkzf4" event={"ID":"9dbf7bb9-a19a-4279-ae78-0e0129e976a7","Type":"ContainerDied","Data":"56b4a746ad2a3096c78737dcaafe805b48d4968198720868fdb08a26b1c3ac2f"} Jun 02 19:04:40.904659 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.904624 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" event={"ID":"85509742-48ce-41fc-a98c-e8c35c87b828","Type":"ContainerStarted","Data":"3913691e95ba6fb996d6a782661d33237c6360016d97faf1f26cb538a857560e"} Jun 02 19:04:40.912398 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.912346 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerStarted","Data":"2b03f87af10ff74ee8fb31a6d0048ad72edfaaa1bc47f0bdcd0903f98f694a2f"} Jun 02 19:04:40.938416 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.937600 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-t5v5v" podStartSLOduration=2.945621154 podStartE2EDuration="3.937584849s" podCreationTimestamp="2026-06-02 19:04:37 +0000 UTC" firstStartedPulling="2026-06-02 19:04:39.428619937 +0000 UTC m=+42.246589640" lastFinishedPulling="2026-06-02 19:04:40.420583633 +0000 UTC m=+43.238553335" observedRunningTime="2026-06-02 19:04:40.936749307 +0000 UTC m=+43.754719033" watchObservedRunningTime="2026-06-02 19:04:40.937584849 +0000 UTC m=+43.755554573" Jun 02 19:04:40.985787 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.983909 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.985787 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.983971 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.985787 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.984064 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-tls\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.985787 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.984091 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.985787 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.984120 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-grpc-tls\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.985787 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.984151 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dpx4l\" (UniqueName: \"kubernetes.io/projected/328db247-e97e-4842-85b2-073684347027-kube-api-access-dpx4l\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.985787 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.984248 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.985787 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.984282 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/328db247-e97e-4842-85b2-073684347027-metrics-client-ca\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.985787 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.984966 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/328db247-e97e-4842-85b2-073684347027-metrics-client-ca\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.987969 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.987925 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.988304 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.988235 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.991034 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.991011 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.991461 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.991424 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.992013 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.991955 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-thanos-querier-tls\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.992772 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.992749 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/328db247-e97e-4842-85b2-073684347027-secret-grpc-tls\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:40.998999 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:40.998909 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpx4l\" (UniqueName: \"kubernetes.io/projected/328db247-e97e-4842-85b2-073684347027-kube-api-access-dpx4l\") pod \"thanos-querier-689cf468df-sh2kv\" (UID: \"328db247-e97e-4842-85b2-073684347027\") " pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:41.038530 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:41.038154 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:42.151796 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.151761 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6fd9645f5f-p58r4"] Jun 02 19:04:42.154697 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.154667 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.157128 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.157108 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 02 19:04:42.157128 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.157117 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 19:04:42.158068 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.158048 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 02 19:04:42.158068 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.158061 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 02 19:04:42.158276 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.158073 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8sked44rrvais\"" Jun 02 19:04:42.158276 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.158187 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-wft7p\"" Jun 02 19:04:42.164761 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.164741 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fd9645f5f-p58r4"] Jun 02 19:04:42.296395 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.296359 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/10f50c19-0858-4245-a3d1-9ae7d69632ea-metrics-server-audit-profiles\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.296598 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.296423 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgvmh\" (UniqueName: \"kubernetes.io/projected/10f50c19-0858-4245-a3d1-9ae7d69632ea-kube-api-access-pgvmh\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.296598 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.296464 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/10f50c19-0858-4245-a3d1-9ae7d69632ea-secret-metrics-server-tls\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.296598 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.296499 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/10f50c19-0858-4245-a3d1-9ae7d69632ea-client-ca-bundle\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.296733 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.296611 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f50c19-0858-4245-a3d1-9ae7d69632ea-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.296733 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.296672 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/10f50c19-0858-4245-a3d1-9ae7d69632ea-audit-log\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.296733 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.296704 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/10f50c19-0858-4245-a3d1-9ae7d69632ea-secret-metrics-server-client-certs\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.398486 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.397777 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/10f50c19-0858-4245-a3d1-9ae7d69632ea-metrics-server-audit-profiles\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.398486 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.397838 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pgvmh\" (UniqueName: \"kubernetes.io/projected/10f50c19-0858-4245-a3d1-9ae7d69632ea-kube-api-access-pgvmh\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.398486 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.397870 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/10f50c19-0858-4245-a3d1-9ae7d69632ea-secret-metrics-server-tls\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.398486 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.397916 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/10f50c19-0858-4245-a3d1-9ae7d69632ea-client-ca-bundle\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.398486 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.397955 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f50c19-0858-4245-a3d1-9ae7d69632ea-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.398486 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.397985 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/10f50c19-0858-4245-a3d1-9ae7d69632ea-audit-log\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.398486 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.398009 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/10f50c19-0858-4245-a3d1-9ae7d69632ea-secret-metrics-server-client-certs\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.398934 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.398854 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/10f50c19-0858-4245-a3d1-9ae7d69632ea-audit-log\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.399184 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.399155 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f50c19-0858-4245-a3d1-9ae7d69632ea-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.399663 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.399620 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/10f50c19-0858-4245-a3d1-9ae7d69632ea-metrics-server-audit-profiles\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.401844 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.401772 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/10f50c19-0858-4245-a3d1-9ae7d69632ea-secret-metrics-server-tls\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.401927 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.401908 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/10f50c19-0858-4245-a3d1-9ae7d69632ea-client-ca-bundle\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.402646 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.402622 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/10f50c19-0858-4245-a3d1-9ae7d69632ea-secret-metrics-server-client-certs\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.411008 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.410983 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgvmh\" (UniqueName: \"kubernetes.io/projected/10f50c19-0858-4245-a3d1-9ae7d69632ea-kube-api-access-pgvmh\") pod \"metrics-server-6fd9645f5f-p58r4\" (UID: \"10f50c19-0858-4245-a3d1-9ae7d69632ea\") " pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.414835 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.414815 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh"] Jun 02 19:04:42.419532 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.419516 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh" Jun 02 19:04:42.422943 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.422856 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 02 19:04:42.422943 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.422868 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-rpzmv\"" Jun 02 19:04:42.441496 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.441471 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh"] Jun 02 19:04:42.466066 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.466045 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:04:42.600278 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.600245 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab780fa2-b9b4-427f-a7a6-fe22af8479af-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-wcmhh\" (UID: \"ab780fa2-b9b4-427f-a7a6-fe22af8479af\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh" Jun 02 19:04:42.701895 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.701815 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab780fa2-b9b4-427f-a7a6-fe22af8479af-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-wcmhh\" (UID: \"ab780fa2-b9b4-427f-a7a6-fe22af8479af\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh" Jun 02 19:04:42.702039 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:42.701989 2574 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 02 19:04:42.702078 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:04:42.702059 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab780fa2-b9b4-427f-a7a6-fe22af8479af-monitoring-plugin-cert podName:ab780fa2-b9b4-427f-a7a6-fe22af8479af nodeName:}" failed. No retries permitted until 2026-06-02 19:04:43.202038965 +0000 UTC m=+46.020008674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/ab780fa2-b9b4-427f-a7a6-fe22af8479af-monitoring-plugin-cert") pod "monitoring-plugin-5768fcc9dd-wcmhh" (UID: "ab780fa2-b9b4-427f-a7a6-fe22af8479af") : secret "monitoring-plugin-cert" not found Jun 02 19:04:42.867794 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.867718 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-747f495d59-8xkqf"] Jun 02 19:04:42.874003 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.873916 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:42.876672 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.876445 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 02 19:04:42.876672 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.876473 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 02 19:04:42.876672 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.876522 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 02 19:04:42.876672 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.876562 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 02 19:04:42.877133 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.877106 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-ll74d\"" Jun 02 19:04:42.877263 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.877230 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 02 19:04:42.884831 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.884566 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-747f495d59-8xkqf"] Jun 02 19:04:42.885580 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:42.885561 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 02 19:04:43.005541 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.005460 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8887c\" (UniqueName: \"kubernetes.io/projected/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-kube-api-access-8887c\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.005672 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.005545 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.005672 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.005620 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-secret-telemeter-client\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.005793 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.005731 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-serving-certs-ca-bundle\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.005793 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.005777 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-federate-client-tls\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.005897 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.005872 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-metrics-client-ca\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.005952 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.005900 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-telemeter-client-tls\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.005952 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.005918 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.107327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.107286 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-serving-certs-ca-bundle\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.107491 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.107451 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-federate-client-tls\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.107562 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.107511 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-metrics-client-ca\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.107562 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.107529 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-telemeter-client-tls\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.107562 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.107546 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.107717 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.107578 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8887c\" (UniqueName: \"kubernetes.io/projected/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-kube-api-access-8887c\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.107717 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.107610 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.107717 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.107631 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-secret-telemeter-client\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.108047 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.108021 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-serving-certs-ca-bundle\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.109108 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.109061 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.109697 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.109654 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-metrics-client-ca\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.111233 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.111171 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-federate-client-tls\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.111341 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.111243 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-telemeter-client-tls\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.111504 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.111475 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.112405 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.112383 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-secret-telemeter-client\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.118689 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.118670 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8887c\" (UniqueName: \"kubernetes.io/projected/5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c-kube-api-access-8887c\") pod \"telemeter-client-747f495d59-8xkqf\" (UID: \"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c\") " pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.186452 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.186425 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" Jun 02 19:04:43.208798 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.208770 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab780fa2-b9b4-427f-a7a6-fe22af8479af-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-wcmhh\" (UID: \"ab780fa2-b9b4-427f-a7a6-fe22af8479af\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh" Jun 02 19:04:43.211176 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.211149 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab780fa2-b9b4-427f-a7a6-fe22af8479af-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-wcmhh\" (UID: \"ab780fa2-b9b4-427f-a7a6-fe22af8479af\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh" Jun 02 19:04:43.330760 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.330695 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh" Jun 02 19:04:43.800634 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.799554 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:04:43.828990 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.828804 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:04:43.829174 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.829001 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:43.837409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.836133 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 02 19:04:43.837409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.836434 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 02 19:04:43.837409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.836742 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 02 19:04:43.837409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.837004 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 02 19:04:43.837409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.837260 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-93fimpkgj6mqn\"" Jun 02 19:04:43.837933 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.837486 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2kvqz\"" Jun 02 19:04:43.837933 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.837536 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 02 19:04:43.837933 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.837689 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 02 19:04:43.837933 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.837797 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 02 19:04:43.838123 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.837945 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 02 19:04:43.840637 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.838190 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 02 19:04:43.840637 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.839236 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 02 19:04:43.840637 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.839335 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 02 19:04:43.841099 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.840686 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 02 19:04:43.852384 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.852326 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-689cf468df-sh2kv"] Jun 02 19:04:43.867027 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:43.866200 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328db247_e97e_4842_85b2_073684347027.slice/crio-ecdce2710fecf1424630fa0f38d6740a498e3959bea3b1b7b248f900eeed1e82 WatchSource:0}: Error finding container ecdce2710fecf1424630fa0f38d6740a498e3959bea3b1b7b248f900eeed1e82: Status 404 returned error can't find the container with id ecdce2710fecf1424630fa0f38d6740a498e3959bea3b1b7b248f900eeed1e82 Jun 02 19:04:43.885720 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.884809 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fd9645f5f-p58r4"] Jun 02 19:04:43.934355 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.931693 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerStarted","Data":"3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb"} Jun 02 19:04:43.934355 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.933585 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" event={"ID":"10f50c19-0858-4245-a3d1-9ae7d69632ea","Type":"ContainerStarted","Data":"d75353ba0a47886f50874c1ae1d3ea0780962e4130f6da19dabdc61beeaeb1e3"} Jun 02 19:04:43.940403 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.940355 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wkzf4" event={"ID":"9dbf7bb9-a19a-4279-ae78-0e0129e976a7","Type":"ContainerStarted","Data":"115341d9ce795960930e6dc2082364932352b88bb678c90feaa736b62b1ec51d"} Jun 02 19:04:43.948090 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.945065 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-747f495d59-8xkqf"] Jun 02 19:04:43.948090 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.947071 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" event={"ID":"febbc43c-abfd-4dee-99f6-4d33cdc29c37","Type":"ContainerStarted","Data":"bbbf182816cf53d80c9d77b7e8180c9b8c00c1e45b7fc221258f5fd77461d9e0"} Jun 02 19:04:43.948090 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.947729 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh"] Jun 02 19:04:43.948460 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:43.948435 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b1ee5ef_21d7_438a_b3d5_e8ed0c5f124c.slice/crio-9fe105db652a09bbfabb626bd2dbcc8c53ebeb73e587eee29f317727a237d666 WatchSource:0}: Error finding container 9fe105db652a09bbfabb626bd2dbcc8c53ebeb73e587eee29f317727a237d666: Status 404 returned error can't find the container with id 9fe105db652a09bbfabb626bd2dbcc8c53ebeb73e587eee29f317727a237d666 Jun 02 19:04:43.951409 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:43.950579 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" event={"ID":"328db247-e97e-4842-85b2-073684347027","Type":"ContainerStarted","Data":"ecdce2710fecf1424630fa0f38d6740a498e3959bea3b1b7b248f900eeed1e82"} Jun 02 19:04:43.956118 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:43.956045 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab780fa2_b9b4_427f_a7a6_fe22af8479af.slice/crio-a4eec5efd5fda7042b872911dca7bc8f375d3b267389dc83f57f833d19b911aa WatchSource:0}: Error finding container a4eec5efd5fda7042b872911dca7bc8f375d3b267389dc83f57f833d19b911aa: Status 404 returned error can't find the container with id a4eec5efd5fda7042b872911dca7bc8f375d3b267389dc83f57f833d19b911aa Jun 02 19:04:44.023488 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.023341 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.023488 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.023402 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.023916 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.023738 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.023916 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.023778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.023916 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.023806 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.023916 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.023841 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.023916 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.023894 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.024253 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.024150 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.024253 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.024178 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.024253 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.024195 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.024912 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.024786 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.024912 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.024856 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.024912 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.024889 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.025123 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.024974 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.025123 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.025001 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.025123 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.025040 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.025123 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.025084 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2hp2\" (UniqueName: \"kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-kube-api-access-n2hp2\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.025123 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.025109 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126191 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126161 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126325 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126195 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126325 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126227 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126325 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126253 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n2hp2\" (UniqueName: \"kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-kube-api-access-n2hp2\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126325 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126279 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126522 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126323 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126522 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126384 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126522 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126432 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126522 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126458 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126522 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126485 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126522 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126520 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126784 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126544 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126784 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126574 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126784 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126603 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126784 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126610 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126784 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126633 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126784 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126670 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126784 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126719 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.126784 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.126753 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.127810 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.127478 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.128372 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.128089 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.128372 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.128364 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.128520 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.128412 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.129309 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.129281 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.132817 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.132775 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.133346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.133300 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.133967 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.133921 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.134062 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.133994 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.134130 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.134114 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.134576 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.134553 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.134643 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.134590 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.135060 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.135024 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.135060 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.135028 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.135497 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.135477 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.136416 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.136394 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.140473 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.140443 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2hp2\" (UniqueName: \"kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-kube-api-access-n2hp2\") pod \"prometheus-k8s-0\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.157036 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.157017 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:44.305026 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.304832 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:04:44.307373 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:44.307344 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f8ebd4_9251_4b2e_95ae_a54845dab544.slice/crio-8db49e99dafbf7f2ce8a71132d392838f36ca015adcb266ec964a5307b7d7fee WatchSource:0}: Error finding container 8db49e99dafbf7f2ce8a71132d392838f36ca015adcb266ec964a5307b7d7fee: Status 404 returned error can't find the container with id 8db49e99dafbf7f2ce8a71132d392838f36ca015adcb266ec964a5307b7d7fee Jun 02 19:04:44.959268 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.958850 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wkzf4" event={"ID":"9dbf7bb9-a19a-4279-ae78-0e0129e976a7","Type":"ContainerStarted","Data":"1d76d2244f015e808c35c35ea70ad0dc14dac29e8478d4f766c5a57ab24fce86"} Jun 02 19:04:44.964316 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.963790 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" event={"ID":"febbc43c-abfd-4dee-99f6-4d33cdc29c37","Type":"ContainerStarted","Data":"cb8d281fb03e7736181cea4160c84210cfab9bfe7a98e778c9895c23462eaea6"} Jun 02 19:04:44.964316 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.963823 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" event={"ID":"febbc43c-abfd-4dee-99f6-4d33cdc29c37","Type":"ContainerStarted","Data":"8c10bd638ea084ddea95c8f91735cf5bff91644567e913727d9e1470b83b50db"} Jun 02 19:04:44.966306 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.966276 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" event={"ID":"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c","Type":"ContainerStarted","Data":"9fe105db652a09bbfabb626bd2dbcc8c53ebeb73e587eee29f317727a237d666"} Jun 02 19:04:44.969424 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.968607 2574 generic.go:358] "Generic (PLEG): container finished" podID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerID="3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb" exitCode=0 Jun 02 19:04:44.969424 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.968671 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerDied","Data":"3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb"} Jun 02 19:04:44.973346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.972549 2574 generic.go:358] "Generic (PLEG): container finished" podID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerID="35dfd83e329febce11e030416c0bf22fdf1d8a2b3deae9710e66f3824f9a5e32" exitCode=0 Jun 02 19:04:44.973346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.972626 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerDied","Data":"35dfd83e329febce11e030416c0bf22fdf1d8a2b3deae9710e66f3824f9a5e32"} Jun 02 19:04:44.973346 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.972646 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerStarted","Data":"8db49e99dafbf7f2ce8a71132d392838f36ca015adcb266ec964a5307b7d7fee"} Jun 02 19:04:44.977700 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.975895 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b95tp" event={"ID":"c641d72d-ffca-480e-9fa8-4b7200759721","Type":"ContainerStarted","Data":"1949c94eb85f3dd18cd88df4bbeaa19226428d8896c56e9874b75894e36a2f3d"} Jun 02 19:04:44.977700 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.977572 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wkzf4" podStartSLOduration=6.70410294 podStartE2EDuration="7.977558886s" podCreationTimestamp="2026-06-02 19:04:37 +0000 UTC" firstStartedPulling="2026-06-02 19:04:38.485347326 +0000 UTC m=+41.303317027" lastFinishedPulling="2026-06-02 19:04:39.758803262 +0000 UTC m=+42.576772973" observedRunningTime="2026-06-02 19:04:44.977177807 +0000 UTC m=+47.795147533" watchObservedRunningTime="2026-06-02 19:04:44.977558886 +0000 UTC m=+47.795528612" Jun 02 19:04:44.980319 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:44.980287 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh" event={"ID":"ab780fa2-b9b4-427f-a7a6-fe22af8479af","Type":"ContainerStarted","Data":"a4eec5efd5fda7042b872911dca7bc8f375d3b267389dc83f57f833d19b911aa"} Jun 02 19:04:45.028839 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:45.028759 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-6jsl7" podStartSLOduration=4.089188938 podStartE2EDuration="8.028742408s" podCreationTimestamp="2026-06-02 19:04:37 +0000 UTC" firstStartedPulling="2026-06-02 19:04:39.738168949 +0000 UTC m=+42.556138651" lastFinishedPulling="2026-06-02 19:04:43.677722417 +0000 UTC m=+46.495692121" observedRunningTime="2026-06-02 19:04:45.027519239 +0000 UTC m=+47.845488966" watchObservedRunningTime="2026-06-02 19:04:45.028742408 +0000 UTC m=+47.846712138" Jun 02 19:04:45.042523 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:45.042290 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b95tp" podStartSLOduration=10.036001851 podStartE2EDuration="16.042276755s" podCreationTimestamp="2026-06-02 19:04:29 +0000 UTC" firstStartedPulling="2026-06-02 19:04:37.678177165 +0000 UTC m=+40.496146867" lastFinishedPulling="2026-06-02 19:04:43.684452053 +0000 UTC m=+46.502421771" observedRunningTime="2026-06-02 19:04:45.040184223 +0000 UTC m=+47.858153959" watchObservedRunningTime="2026-06-02 19:04:45.042276755 +0000 UTC m=+47.860246480" Jun 02 19:04:48.896559 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:48.896531 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-26xk6" Jun 02 19:04:49.021131 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.021017 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-779c6f5b94-bc5r7"] Jun 02 19:04:49.041778 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.041720 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-779c6f5b94-bc5r7"] Jun 02 19:04:49.041778 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.041773 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.044960 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.044937 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 02 19:04:49.045093 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.044970 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 02 19:04:49.045093 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.044945 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 02 19:04:49.045093 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.044944 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 02 19:04:49.045272 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.045257 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 02 19:04:49.045326 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.045311 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8xmml\"" Jun 02 19:04:49.081303 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.081281 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-config\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.081434 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.081395 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-oauth-config\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.081528 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.081504 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-serving-cert\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.081602 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.081541 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-oauth-serving-cert\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.081602 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.081566 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpqwb\" (UniqueName: \"kubernetes.io/projected/4dea0451-37c1-46d2-8532-a8ec8427f3c3-kube-api-access-kpqwb\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.081708 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.081626 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-service-ca\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.182815 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.182751 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-serving-cert\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.182815 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.182793 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-oauth-serving-cert\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.182972 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.182819 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kpqwb\" (UniqueName: \"kubernetes.io/projected/4dea0451-37c1-46d2-8532-a8ec8427f3c3-kube-api-access-kpqwb\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.182972 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.182894 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-service-ca\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.182972 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.182943 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-config\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.183149 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.183124 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-oauth-config\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.183652 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.183632 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-config\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.183759 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.183640 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-oauth-serving-cert\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.183759 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.183674 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-service-ca\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.186032 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.186001 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-oauth-config\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.186128 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.186100 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-serving-cert\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.191146 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.191123 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpqwb\" (UniqueName: \"kubernetes.io/projected/4dea0451-37c1-46d2-8532-a8ec8427f3c3-kube-api-access-kpqwb\") pod \"console-779c6f5b94-bc5r7\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:49.354527 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:49.354493 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:04:50.354375 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:50.354285 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-779c6f5b94-bc5r7"] Jun 02 19:04:50.359230 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:04:50.359199 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dea0451_37c1_46d2_8532_a8ec8427f3c3.slice/crio-ad6dff27f7387f3a147d56edec18e615f2984752cb0e83c1e87265633292e701 WatchSource:0}: Error finding container ad6dff27f7387f3a147d56edec18e615f2984752cb0e83c1e87265633292e701: Status 404 returned error can't find the container with id ad6dff27f7387f3a147d56edec18e615f2984752cb0e83c1e87265633292e701 Jun 02 19:04:51.004068 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.004041 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh" event={"ID":"ab780fa2-b9b4-427f-a7a6-fe22af8479af","Type":"ContainerStarted","Data":"99b4d33830feaef188476647f22a207c5cf6745ddaffe743719bca6dfeff4a4d"} Jun 02 19:04:51.006272 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.004660 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh" Jun 02 19:04:51.006272 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.006236 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" event={"ID":"10f50c19-0858-4245-a3d1-9ae7d69632ea","Type":"ContainerStarted","Data":"f799019dbfe3b02dc066cfbf89886736e6864a7c1c6b045d6205a5c7480d79bb"} Jun 02 19:04:51.008388 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.008359 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779c6f5b94-bc5r7" event={"ID":"4dea0451-37c1-46d2-8532-a8ec8427f3c3","Type":"ContainerStarted","Data":"ad6dff27f7387f3a147d56edec18e615f2984752cb0e83c1e87265633292e701"} Jun 02 19:04:51.010971 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.010953 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh" Jun 02 19:04:51.012020 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.011998 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" event={"ID":"328db247-e97e-4842-85b2-073684347027","Type":"ContainerStarted","Data":"e00b2de9d9f4dc78524fa9b5044ca279a049c89ffa6aadabc36477e90740e6af"} Jun 02 19:04:51.012115 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.012025 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" event={"ID":"328db247-e97e-4842-85b2-073684347027","Type":"ContainerStarted","Data":"32e2d3c82c5f2a5b46ae9ec18a1562e0d88132ea8db4d482e06d18b53ef2e1c9"} Jun 02 19:04:51.012115 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.012036 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" event={"ID":"328db247-e97e-4842-85b2-073684347027","Type":"ContainerStarted","Data":"6e9cedec0c7806f2252002021950d40a04e40c63f668c968c33e900ec8614ac6"} Jun 02 19:04:51.014049 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.014022 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" event={"ID":"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c","Type":"ContainerStarted","Data":"5a7035ef0de5a28770c7dae0e5a04c2934dd12dfc23b70fb774be1644d273299"} Jun 02 19:04:51.014138 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.014053 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" event={"ID":"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c","Type":"ContainerStarted","Data":"7aad255f9c44bc8f3935958c0cc256b8ced32d25a113d3644e7ef12d8f085d2f"} Jun 02 19:04:51.014138 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.014068 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" event={"ID":"5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c","Type":"ContainerStarted","Data":"1cc259429404f3757b618d451143195968c44639cd51e2440cec130adf2ed1fb"} Jun 02 19:04:51.017401 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.017381 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerStarted","Data":"7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241"} Jun 02 19:04:51.017505 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.017408 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerStarted","Data":"2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5"} Jun 02 19:04:51.017505 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.017422 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerStarted","Data":"a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47"} Jun 02 19:04:51.017505 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.017433 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerStarted","Data":"6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d"} Jun 02 19:04:51.017505 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.017445 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerStarted","Data":"ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd"} Jun 02 19:04:51.021571 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.021554 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerStarted","Data":"8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911"} Jun 02 19:04:51.021640 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.021575 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerStarted","Data":"0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984"} Jun 02 19:04:51.021640 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.021585 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerStarted","Data":"832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259"} Jun 02 19:04:51.021640 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.021593 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerStarted","Data":"04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96"} Jun 02 19:04:51.021640 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.021601 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerStarted","Data":"37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65"} Jun 02 19:04:51.036672 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.036623 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-wcmhh" podStartSLOduration=2.817317003 podStartE2EDuration="9.036609578s" podCreationTimestamp="2026-06-02 19:04:42 +0000 UTC" firstStartedPulling="2026-06-02 19:04:43.958572285 +0000 UTC m=+46.776541986" lastFinishedPulling="2026-06-02 19:04:50.177864844 +0000 UTC m=+52.995834561" observedRunningTime="2026-06-02 19:04:51.019675389 +0000 UTC m=+53.837645115" watchObservedRunningTime="2026-06-02 19:04:51.036609578 +0000 UTC m=+53.854579305" Jun 02 19:04:51.037523 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.037479 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" podStartSLOduration=2.784602761 podStartE2EDuration="9.037471614s" podCreationTimestamp="2026-06-02 19:04:42 +0000 UTC" firstStartedPulling="2026-06-02 19:04:43.924997123 +0000 UTC m=+46.742966825" lastFinishedPulling="2026-06-02 19:04:50.17786597 +0000 UTC m=+52.995835678" observedRunningTime="2026-06-02 19:04:51.035293204 +0000 UTC m=+53.853262929" watchObservedRunningTime="2026-06-02 19:04:51.037471614 +0000 UTC m=+53.855441337" Jun 02 19:04:51.057725 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:51.057686 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-747f495d59-8xkqf" podStartSLOduration=3.213033335 podStartE2EDuration="9.057673657s" podCreationTimestamp="2026-06-02 19:04:42 +0000 UTC" firstStartedPulling="2026-06-02 19:04:43.951015419 +0000 UTC m=+46.768985123" lastFinishedPulling="2026-06-02 19:04:49.795655728 +0000 UTC m=+52.613625445" observedRunningTime="2026-06-02 19:04:51.056371527 +0000 UTC m=+53.874341244" watchObservedRunningTime="2026-06-02 19:04:51.057673657 +0000 UTC m=+53.875643391" Jun 02 19:04:52.029646 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:52.029607 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerStarted","Data":"b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847"} Jun 02 19:04:52.083587 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:52.082465 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.838933303 podStartE2EDuration="9.082447627s" podCreationTimestamp="2026-06-02 19:04:43 +0000 UTC" firstStartedPulling="2026-06-02 19:04:44.974061181 +0000 UTC m=+47.792030886" lastFinishedPulling="2026-06-02 19:04:50.217575489 +0000 UTC m=+53.035545210" observedRunningTime="2026-06-02 19:04:52.07920744 +0000 UTC m=+54.897177165" watchObservedRunningTime="2026-06-02 19:04:52.082447627 +0000 UTC m=+54.900417355" Jun 02 19:04:53.036999 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:53.036905 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" event={"ID":"328db247-e97e-4842-85b2-073684347027","Type":"ContainerStarted","Data":"a1b3dad0b5ea356f52e13c0f455e6d3d3fd916a72fc7e4e9d7d5ea53c1dbe60f"} Jun 02 19:04:53.036999 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:53.036945 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" event={"ID":"328db247-e97e-4842-85b2-073684347027","Type":"ContainerStarted","Data":"f70b90cd21d31aa013c6681d6de115b25ff4d25a860b361b2f32093468826bef"} Jun 02 19:04:53.036999 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:53.036959 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" event={"ID":"328db247-e97e-4842-85b2-073684347027","Type":"ContainerStarted","Data":"4e3f184d29da8c91a8a38957eb1213051a8640045b839bfbbfe3626f04d5d949"} Jun 02 19:04:53.037574 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:53.037353 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:04:53.040580 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:53.040537 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerStarted","Data":"9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b"} Jun 02 19:04:53.063207 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:53.062456 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" podStartSLOduration=4.873371022 podStartE2EDuration="13.062440503s" podCreationTimestamp="2026-06-02 19:04:40 +0000 UTC" firstStartedPulling="2026-06-02 19:04:43.868888985 +0000 UTC m=+46.686858703" lastFinishedPulling="2026-06-02 19:04:52.057958469 +0000 UTC m=+54.875928184" observedRunningTime="2026-06-02 19:04:53.060332099 +0000 UTC m=+55.878301832" watchObservedRunningTime="2026-06-02 19:04:53.062440503 +0000 UTC m=+55.880410227" Jun 02 19:04:53.086611 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:53.086561 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.917856393 podStartE2EDuration="15.086548262s" podCreationTimestamp="2026-06-02 19:04:38 +0000 UTC" firstStartedPulling="2026-06-02 19:04:39.891428783 +0000 UTC m=+42.709398499" lastFinishedPulling="2026-06-02 19:04:52.060120652 +0000 UTC m=+54.878090368" observedRunningTime="2026-06-02 19:04:53.085825905 +0000 UTC m=+55.903795632" watchObservedRunningTime="2026-06-02 19:04:53.086548262 +0000 UTC m=+55.904517987" Jun 02 19:04:54.158512 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.158477 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:04:54.787507 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.787473 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-8567df9dc9-k5f7r"] Jun 02 19:04:54.810279 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.810252 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8567df9dc9-k5f7r"] Jun 02 19:04:54.810433 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.810367 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.816833 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.816810 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 02 19:04:54.838641 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.838612 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-oauth-config\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.838767 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.838657 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-console-config\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.838767 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.838734 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-service-ca\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.838849 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.838818 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-oauth-serving-cert\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.838889 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.838855 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-serving-cert\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.838889 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.838879 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-trusted-ca-bundle\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.838988 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.838946 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw74b\" (UniqueName: \"kubernetes.io/projected/03f3828b-2b66-429f-a69c-753c863bd02f-kube-api-access-mw74b\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.939718 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.939683 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-oauth-serving-cert\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.939718 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.939722 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-serving-cert\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.939925 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.939746 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-trusted-ca-bundle\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.939925 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.939811 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mw74b\" (UniqueName: \"kubernetes.io/projected/03f3828b-2b66-429f-a69c-753c863bd02f-kube-api-access-mw74b\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.939925 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.939887 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-oauth-config\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.939925 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.939917 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-console-config\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.940157 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.939982 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-service-ca\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.940536 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.940501 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-oauth-serving-cert\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.940650 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.940550 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-trusted-ca-bundle\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.940714 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.940696 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-service-ca\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.940772 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.940730 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-console-config\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.942564 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.942546 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-oauth-config\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.942823 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.942799 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-serving-cert\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:54.948606 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:54.948586 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw74b\" (UniqueName: \"kubernetes.io/projected/03f3828b-2b66-429f-a69c-753c863bd02f-kube-api-access-mw74b\") pod \"console-8567df9dc9-k5f7r\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:55.122737 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:55.122656 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:04:55.844576 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:55.844549 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4mqnl" Jun 02 19:04:57.873357 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:57.873332 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f7fdc8998-tqs6k" Jun 02 19:04:59.052314 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:04:59.052284 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-689cf468df-sh2kv" Jun 02 19:05:01.889339 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:01.889313 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8567df9dc9-k5f7r"] Jun 02 19:05:01.890965 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:05:01.890946 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f3828b_2b66_429f_a69c_753c863bd02f.slice/crio-7d55815bbcc14a71b15c9c9ffdad2e9056182b38c33ec4df47457bd4d1df122a WatchSource:0}: Error finding container 7d55815bbcc14a71b15c9c9ffdad2e9056182b38c33ec4df47457bd4d1df122a: Status 404 returned error can't find the container with id 7d55815bbcc14a71b15c9c9ffdad2e9056182b38c33ec4df47457bd4d1df122a Jun 02 19:05:02.072655 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.072574 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-6q4kq" event={"ID":"e25f59c3-6117-4dbb-8dc5-dd26b49d43f8","Type":"ContainerStarted","Data":"de0de533174f58fe7022de4ed193a58d31d43ea409c670aa112ed537108dbca9"} Jun 02 19:05:02.072795 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.072773 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-6q4kq" Jun 02 19:05:02.073992 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.073962 2574 patch_prober.go:28] interesting pod/downloads-c65b8dc4b-6q4kq container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.12:8080/\": dial tcp 10.133.0.12:8080: connect: connection refused" start-of-body= Jun 02 19:05:02.074138 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.074017 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-c65b8dc4b-6q4kq" podUID="e25f59c3-6117-4dbb-8dc5-dd26b49d43f8" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.12:8080/\": dial tcp 10.133.0.12:8080: connect: connection refused" Jun 02 19:05:02.074386 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.074360 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8567df9dc9-k5f7r" event={"ID":"03f3828b-2b66-429f-a69c-753c863bd02f","Type":"ContainerStarted","Data":"8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a"} Jun 02 19:05:02.074509 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.074393 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8567df9dc9-k5f7r" event={"ID":"03f3828b-2b66-429f-a69c-753c863bd02f","Type":"ContainerStarted","Data":"7d55815bbcc14a71b15c9c9ffdad2e9056182b38c33ec4df47457bd4d1df122a"} Jun 02 19:05:02.075941 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.075913 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779c6f5b94-bc5r7" event={"ID":"4dea0451-37c1-46d2-8532-a8ec8427f3c3","Type":"ContainerStarted","Data":"cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb"} Jun 02 19:05:02.089787 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.089702 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-6q4kq" podStartSLOduration=1.846906724 podStartE2EDuration="24.089687446s" podCreationTimestamp="2026-06-02 19:04:38 +0000 UTC" firstStartedPulling="2026-06-02 19:04:39.525814284 +0000 UTC m=+42.343783992" lastFinishedPulling="2026-06-02 19:05:01.768594999 +0000 UTC m=+64.586564714" observedRunningTime="2026-06-02 19:05:02.088121752 +0000 UTC m=+64.906091477" watchObservedRunningTime="2026-06-02 19:05:02.089687446 +0000 UTC m=+64.907657170" Jun 02 19:05:02.105442 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.105405 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-779c6f5b94-bc5r7" podStartSLOduration=2.699366326 podStartE2EDuration="14.105393684s" podCreationTimestamp="2026-06-02 19:04:48 +0000 UTC" firstStartedPulling="2026-06-02 19:04:50.361635368 +0000 UTC m=+53.179605070" lastFinishedPulling="2026-06-02 19:05:01.767662706 +0000 UTC m=+64.585632428" observedRunningTime="2026-06-02 19:05:02.104744215 +0000 UTC m=+64.922713939" watchObservedRunningTime="2026-06-02 19:05:02.105393684 +0000 UTC m=+64.923363412" Jun 02 19:05:02.122477 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.122441 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-8567df9dc9-k5f7r" podStartSLOduration=8.122431242 podStartE2EDuration="8.122431242s" podCreationTimestamp="2026-06-02 19:04:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:05:02.121281435 +0000 UTC m=+64.939251160" watchObservedRunningTime="2026-06-02 19:05:02.122431242 +0000 UTC m=+64.940400966" Jun 02 19:05:02.466272 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.466233 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:05:02.466543 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:02.466287 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:05:03.092831 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.092799 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-6q4kq" Jun 02 19:05:03.323925 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.323885 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:05:03.326123 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.326095 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 19:05:03.337789 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.337685 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e006346-8709-4a02-bf44-d9e5a59bb74b-metrics-certs\") pod \"network-metrics-daemon-mp2fh\" (UID: \"9e006346-8709-4a02-bf44-d9e5a59bb74b\") " pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:05:03.424701 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.424665 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nhn\" (UniqueName: \"kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn\") pod \"network-check-target-xc29k\" (UID: \"ebe8479b-96cc-4fd7-928d-fd02f2345f2a\") " pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:05:03.427157 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.427129 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 19:05:03.437407 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.437382 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 19:05:03.448315 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.448290 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6nhn\" (UniqueName: \"kubernetes.io/projected/ebe8479b-96cc-4fd7-928d-fd02f2345f2a-kube-api-access-k6nhn\") pod \"network-check-target-xc29k\" (UID: \"ebe8479b-96cc-4fd7-928d-fd02f2345f2a\") " pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:05:03.600866 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.600835 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dks2v\"" Jun 02 19:05:03.605076 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.605055 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mp95k\"" Jun 02 19:05:03.609528 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.609507 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp2fh" Jun 02 19:05:03.614263 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.614239 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:05:03.774704 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.774681 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mp2fh"] Jun 02 19:05:03.776823 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:05:03.776765 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e006346_8709_4a02_bf44_d9e5a59bb74b.slice/crio-69860b38eaffb41d4404e968e691ef03f0e6e56020d4d0e17319c0f3385e45f7 WatchSource:0}: Error finding container 69860b38eaffb41d4404e968e691ef03f0e6e56020d4d0e17319c0f3385e45f7: Status 404 returned error can't find the container with id 69860b38eaffb41d4404e968e691ef03f0e6e56020d4d0e17319c0f3385e45f7 Jun 02 19:05:03.791012 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:03.790985 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xc29k"] Jun 02 19:05:03.793769 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:05:03.793728 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe8479b_96cc_4fd7_928d_fd02f2345f2a.slice/crio-c25fe11849be91b072e9da8174e5fc3aa9f15196bc001a55b7d08ec2ba0d50bc WatchSource:0}: Error finding container c25fe11849be91b072e9da8174e5fc3aa9f15196bc001a55b7d08ec2ba0d50bc: Status 404 returned error can't find the container with id c25fe11849be91b072e9da8174e5fc3aa9f15196bc001a55b7d08ec2ba0d50bc Jun 02 19:05:04.084414 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:04.084329 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mp2fh" event={"ID":"9e006346-8709-4a02-bf44-d9e5a59bb74b","Type":"ContainerStarted","Data":"69860b38eaffb41d4404e968e691ef03f0e6e56020d4d0e17319c0f3385e45f7"} Jun 02 19:05:04.085725 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:04.085696 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xc29k" event={"ID":"ebe8479b-96cc-4fd7-928d-fd02f2345f2a","Type":"ContainerStarted","Data":"c25fe11849be91b072e9da8174e5fc3aa9f15196bc001a55b7d08ec2ba0d50bc"} Jun 02 19:05:05.123389 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:05.123347 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:05:05.123860 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:05.123733 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:05:05.131892 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:05.131643 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:05:05.723570 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:05.723537 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:05.780176 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:05.779960 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:06.100946 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:06.100878 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mp2fh" event={"ID":"9e006346-8709-4a02-bf44-d9e5a59bb74b","Type":"ContainerStarted","Data":"4d5738395ae48f04a11d5b66b44da9240559dc991630f221d600ad88f34c05b5"} Jun 02 19:05:06.100946 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:06.100920 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mp2fh" event={"ID":"9e006346-8709-4a02-bf44-d9e5a59bb74b","Type":"ContainerStarted","Data":"04d415c5d7eaeb163de43eadd9f4f85df973924a8cec689ea7601f9aea9b797e"} Jun 02 19:05:06.107600 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:06.107409 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:05:06.121506 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:06.121483 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:06.232168 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:06.232140 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-779c6f5b94-bc5r7"] Jun 02 19:05:07.698754 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:07.698698 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mp2fh" podStartSLOduration=68.730306233 podStartE2EDuration="1m10.698681398s" podCreationTimestamp="2026-06-02 19:03:57 +0000 UTC" firstStartedPulling="2026-06-02 19:05:03.779399655 +0000 UTC m=+66.597369357" lastFinishedPulling="2026-06-02 19:05:05.747774821 +0000 UTC m=+68.565744522" observedRunningTime="2026-06-02 19:05:07.124935884 +0000 UTC m=+69.942905614" watchObservedRunningTime="2026-06-02 19:05:07.698681398 +0000 UTC m=+70.516651123" Jun 02 19:05:08.928488 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:08.928458 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f0d5f2e4-5750-4463-b72f-dd93bb136eee/init-config-reloader/0.log" Jun 02 19:05:09.114956 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:09.114927 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xc29k" event={"ID":"ebe8479b-96cc-4fd7-928d-fd02f2345f2a","Type":"ContainerStarted","Data":"ed2297c46944646dc815425b2090b0afd39549276b95e05847beb77750a52d86"} Jun 02 19:05:09.115177 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:09.114978 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:05:09.131154 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:09.131131 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f0d5f2e4-5750-4463-b72f-dd93bb136eee/alertmanager/0.log" Jun 02 19:05:09.134942 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:09.134898 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xc29k" podStartSLOduration=67.754842089 podStartE2EDuration="1m12.134880983s" podCreationTimestamp="2026-06-02 19:03:57 +0000 UTC" firstStartedPulling="2026-06-02 19:05:03.796081028 +0000 UTC m=+66.614050736" lastFinishedPulling="2026-06-02 19:05:08.176119913 +0000 UTC m=+70.994089630" observedRunningTime="2026-06-02 19:05:09.133797517 +0000 UTC m=+71.951767241" watchObservedRunningTime="2026-06-02 19:05:09.134880983 +0000 UTC m=+71.952850709" Jun 02 19:05:09.331542 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:09.331474 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f0d5f2e4-5750-4463-b72f-dd93bb136eee/config-reloader/0.log" Jun 02 19:05:09.354639 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:09.354611 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:05:09.532996 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:09.532969 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f0d5f2e4-5750-4463-b72f-dd93bb136eee/kube-rbac-proxy-web/0.log" Jun 02 19:05:09.739567 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:09.739542 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f0d5f2e4-5750-4463-b72f-dd93bb136eee/kube-rbac-proxy/0.log" Jun 02 19:05:09.927783 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:09.927756 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f0d5f2e4-5750-4463-b72f-dd93bb136eee/kube-rbac-proxy-metric/0.log" Jun 02 19:05:10.128741 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:10.128676 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f0d5f2e4-5750-4463-b72f-dd93bb136eee/prom-label-proxy/0.log" Jun 02 19:05:10.527721 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:10.527693 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-6jsl7_febbc43c-abfd-4dee-99f6-4d33cdc29c37/kube-state-metrics/0.log" Jun 02 19:05:10.727580 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:10.727550 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-6jsl7_febbc43c-abfd-4dee-99f6-4d33cdc29c37/kube-rbac-proxy-main/0.log" Jun 02 19:05:10.927284 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:10.927258 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-6jsl7_febbc43c-abfd-4dee-99f6-4d33cdc29c37/kube-rbac-proxy-self/0.log" Jun 02 19:05:11.127324 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:11.127305 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6fd9645f5f-p58r4_10f50c19-0858-4245-a3d1-9ae7d69632ea/metrics-server/0.log" Jun 02 19:05:11.330878 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:11.330819 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-wcmhh_ab780fa2-b9b4-427f-a7a6-fe22af8479af/monitoring-plugin/0.log" Jun 02 19:05:12.727993 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:12.727964 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wkzf4_9dbf7bb9-a19a-4279-ae78-0e0129e976a7/init-textfile/0.log" Jun 02 19:05:12.928596 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:12.928571 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wkzf4_9dbf7bb9-a19a-4279-ae78-0e0129e976a7/node-exporter/0.log" Jun 02 19:05:13.127415 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:13.127350 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wkzf4_9dbf7bb9-a19a-4279-ae78-0e0129e976a7/kube-rbac-proxy/0.log" Jun 02 19:05:13.330895 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:13.330868 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-t5v5v_85509742-48ce-41fc-a98c-e8c35c87b828/kube-rbac-proxy-main/0.log" Jun 02 19:05:13.526937 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:13.526918 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-t5v5v_85509742-48ce-41fc-a98c-e8c35c87b828/kube-rbac-proxy-self/0.log" Jun 02 19:05:13.727025 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:13.727003 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-t5v5v_85509742-48ce-41fc-a98c-e8c35c87b828/openshift-state-metrics/0.log" Jun 02 19:05:13.927571 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:13.927551 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0f8ebd4-9251-4b2e-95ae-a54845dab544/init-config-reloader/0.log" Jun 02 19:05:14.129383 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:14.129359 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0f8ebd4-9251-4b2e-95ae-a54845dab544/prometheus/0.log" Jun 02 19:05:14.327863 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:14.327802 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0f8ebd4-9251-4b2e-95ae-a54845dab544/config-reloader/0.log" Jun 02 19:05:14.528165 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:14.528141 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0f8ebd4-9251-4b2e-95ae-a54845dab544/thanos-sidecar/0.log" Jun 02 19:05:14.727666 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:14.727645 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0f8ebd4-9251-4b2e-95ae-a54845dab544/kube-rbac-proxy-web/0.log" Jun 02 19:05:14.927740 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:14.927720 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0f8ebd4-9251-4b2e-95ae-a54845dab544/kube-rbac-proxy/0.log" Jun 02 19:05:15.128014 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:15.127965 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0f8ebd4-9251-4b2e-95ae-a54845dab544/kube-rbac-proxy-thanos/0.log" Jun 02 19:05:15.928059 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:15.928026 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747f495d59-8xkqf_5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c/telemeter-client/0.log" Jun 02 19:05:16.127708 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:16.127689 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747f495d59-8xkqf_5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c/reload/0.log" Jun 02 19:05:16.327290 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:16.327238 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747f495d59-8xkqf_5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c/kube-rbac-proxy/0.log" Jun 02 19:05:16.527777 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:16.527750 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/thanos-query/0.log" Jun 02 19:05:16.727121 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:16.727098 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/kube-rbac-proxy-web/0.log" Jun 02 19:05:16.927952 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:16.927932 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/kube-rbac-proxy/0.log" Jun 02 19:05:17.127589 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:17.127524 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/prom-label-proxy/0.log" Jun 02 19:05:17.327207 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:17.327178 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/kube-rbac-proxy-rules/0.log" Jun 02 19:05:17.528022 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:17.527997 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/kube-rbac-proxy-metrics/0.log" Jun 02 19:05:18.327959 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:18.327930 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-779c6f5b94-bc5r7_4dea0451-37c1-46d2-8532-a8ec8427f3c3/console/0.log" Jun 02 19:05:18.527802 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:18.527773 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8567df9dc9-k5f7r_03f3828b-2b66-429f-a69c-753c863bd02f/console/0.log" Jun 02 19:05:18.728458 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:18.728435 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-6q4kq_e25f59c3-6117-4dbb-8dc5-dd26b49d43f8/download-server/0.log" Jun 02 19:05:19.327231 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:19.327192 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mv67v_5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af/serve-healthcheck-canary/0.log" Jun 02 19:05:22.471749 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:22.471721 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:05:22.475908 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:22.475885 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6fd9645f5f-p58r4" Jun 02 19:05:27.446992 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:27.446935 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:05:27.447439 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:27.447404 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="alertmanager" containerID="cri-o://ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd" gracePeriod=120 Jun 02 19:05:27.447576 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:27.447436 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy" containerID="cri-o://2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5" gracePeriod=120 Jun 02 19:05:27.447576 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:27.447434 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="prom-label-proxy" containerID="cri-o://9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b" gracePeriod=120 Jun 02 19:05:27.447576 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:27.447471 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy-metric" containerID="cri-o://7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241" gracePeriod=120 Jun 02 19:05:27.447576 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:27.447489 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy-web" containerID="cri-o://a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47" gracePeriod=120 Jun 02 19:05:27.447576 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:27.447537 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="config-reloader" containerID="cri-o://6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d" gracePeriod=120 Jun 02 19:05:28.183477 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.183449 2574 generic.go:358] "Generic (PLEG): container finished" podID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerID="9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b" exitCode=0 Jun 02 19:05:28.183477 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.183474 2574 generic.go:358] "Generic (PLEG): container finished" podID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerID="7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241" exitCode=0 Jun 02 19:05:28.183477 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.183481 2574 generic.go:358] "Generic (PLEG): container finished" podID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerID="2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5" exitCode=0 Jun 02 19:05:28.183800 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.183487 2574 generic.go:358] "Generic (PLEG): container finished" podID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerID="6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d" exitCode=0 Jun 02 19:05:28.183800 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.183493 2574 generic.go:358] "Generic (PLEG): container finished" podID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerID="ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd" exitCode=0 Jun 02 19:05:28.183800 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.183527 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerDied","Data":"9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b"} Jun 02 19:05:28.183800 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.183568 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerDied","Data":"7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241"} Jun 02 19:05:28.183800 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.183585 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerDied","Data":"2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5"} Jun 02 19:05:28.183800 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.183601 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerDied","Data":"6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d"} Jun 02 19:05:28.183800 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.183614 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerDied","Data":"ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd"} Jun 02 19:05:28.726265 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.726244 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:05:28.845359 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845291 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-out\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845359 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845324 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-tls-assets\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845581 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845360 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845581 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845388 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-main-tls\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845581 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845430 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-main-db\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845581 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845464 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-web-config\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845581 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845493 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-web\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845581 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845530 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gt68m\" (UniqueName: \"kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-kube-api-access-gt68m\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845581 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845572 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-volume\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845933 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845597 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-metrics-client-ca\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845933 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845628 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-cluster-tls-config\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845933 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845689 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-trusted-ca-bundle\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845933 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845729 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\" (UID: \"f0d5f2e4-5750-4463-b72f-dd93bb136eee\") " Jun 02 19:05:28.845933 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.845780 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:05:28.846245 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.846044 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-main-db\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.846357 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.846318 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:05:28.846976 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.846937 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:05:28.849376 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.849294 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:05:28.849376 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.849290 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:28.849376 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.849357 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-out" (OuterVolumeSpecName: "config-out") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:05:28.849595 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.849434 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:28.849595 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.849547 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-volume" (OuterVolumeSpecName: "config-volume") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:28.849710 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.849684 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:28.849773 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.849720 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-kube-api-access-gt68m" (OuterVolumeSpecName: "kube-api-access-gt68m") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "kube-api-access-gt68m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:05:28.849841 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.849792 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:28.853580 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.853444 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:28.860296 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.860206 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-web-config" (OuterVolumeSpecName: "web-config") pod "f0d5f2e4-5750-4463-b72f-dd93bb136eee" (UID: "f0d5f2e4-5750-4463-b72f-dd93bb136eee"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:28.947331 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947308 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.947455 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947332 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-main-tls\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.947455 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947346 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-web-config\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.947455 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947359 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.947455 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947373 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt68m\" (UniqueName: \"kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-kube-api-access-gt68m\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.947455 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947388 2574 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-volume\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.947455 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947401 2574 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-metrics-client-ca\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.947455 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947416 2574 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-cluster-tls-config\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.947455 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947430 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0d5f2e4-5750-4463-b72f-dd93bb136eee-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.947455 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947445 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f0d5f2e4-5750-4463-b72f-dd93bb136eee-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.947725 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947460 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f0d5f2e4-5750-4463-b72f-dd93bb136eee-config-out\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:28.947725 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:28.947473 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f0d5f2e4-5750-4463-b72f-dd93bb136eee-tls-assets\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:29.189473 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.189449 2574 generic.go:358] "Generic (PLEG): container finished" podID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerID="a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47" exitCode=0 Jun 02 19:05:29.189605 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.189525 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerDied","Data":"a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47"} Jun 02 19:05:29.189605 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.189564 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f0d5f2e4-5750-4463-b72f-dd93bb136eee","Type":"ContainerDied","Data":"2b03f87af10ff74ee8fb31a6d0048ad72edfaaa1bc47f0bdcd0903f98f694a2f"} Jun 02 19:05:29.189605 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.189569 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 19:05:29.189605 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.189580 2574 scope.go:117] "RemoveContainer" containerID="9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b" Jun 02 19:05:29.199863 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.199847 2574 scope.go:117] "RemoveContainer" containerID="7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241" Jun 02 19:05:29.206082 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.206067 2574 scope.go:117] "RemoveContainer" containerID="2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5" Jun 02 19:05:29.212368 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.211948 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:05:29.219458 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.219435 2574 scope.go:117] "RemoveContainer" containerID="a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47" Jun 02 19:05:29.220273 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.220251 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 19:05:29.227727 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.227711 2574 scope.go:117] "RemoveContainer" containerID="6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d" Jun 02 19:05:29.234057 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.234043 2574 scope.go:117] "RemoveContainer" containerID="ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd" Jun 02 19:05:29.240252 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.240228 2574 scope.go:117] "RemoveContainer" containerID="3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb" Jun 02 19:05:29.246431 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.246414 2574 scope.go:117] "RemoveContainer" containerID="9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b" Jun 02 19:05:29.246698 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:29.246678 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b\": container with ID starting with 9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b not found: ID does not exist" containerID="9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b" Jun 02 19:05:29.246759 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.246706 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b"} err="failed to get container status \"9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b\": rpc error: code = NotFound desc = could not find container \"9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b\": container with ID starting with 9689c0c298dd3afa9eabcee17cf5a1240724a515b05328ef742adbebfe652b9b not found: ID does not exist" Jun 02 19:05:29.246759 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.246741 2574 scope.go:117] "RemoveContainer" containerID="7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241" Jun 02 19:05:29.246979 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:29.246959 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241\": container with ID starting with 7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241 not found: ID does not exist" containerID="7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241" Jun 02 19:05:29.247020 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.246985 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241"} err="failed to get container status \"7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241\": rpc error: code = NotFound desc = could not find container \"7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241\": container with ID starting with 7fbcc05106dc0600ee43c2e81e0854ff6d42b2abeaac56dc0d5170697b508241 not found: ID does not exist" Jun 02 19:05:29.247020 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.247002 2574 scope.go:117] "RemoveContainer" containerID="2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5" Jun 02 19:05:29.247199 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:29.247185 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5\": container with ID starting with 2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5 not found: ID does not exist" containerID="2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5" Jun 02 19:05:29.247256 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.247204 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5"} err="failed to get container status \"2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5\": rpc error: code = NotFound desc = could not find container \"2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5\": container with ID starting with 2e55159e68d77e1633f107b976e7e9168b2a8080e57a2f49c91883decd05aee5 not found: ID does not exist" Jun 02 19:05:29.247296 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.247253 2574 scope.go:117] "RemoveContainer" containerID="a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47" Jun 02 19:05:29.247488 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:29.247471 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47\": container with ID starting with a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47 not found: ID does not exist" containerID="a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47" Jun 02 19:05:29.247560 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.247496 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47"} err="failed to get container status \"a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47\": rpc error: code = NotFound desc = could not find container \"a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47\": container with ID starting with a7e4301e3a28d8370c1d18742d5f0b4888b6850900bf7f48fc32206f730eac47 not found: ID does not exist" Jun 02 19:05:29.247560 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.247520 2574 scope.go:117] "RemoveContainer" containerID="6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d" Jun 02 19:05:29.247760 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:29.247744 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d\": container with ID starting with 6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d not found: ID does not exist" containerID="6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d" Jun 02 19:05:29.247810 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.247765 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d"} err="failed to get container status \"6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d\": rpc error: code = NotFound desc = could not find container \"6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d\": container with ID starting with 6fae38adb618b630e71774c8eaf7cfa5c4956dd7d806381420ca8570e86f7e9d not found: ID does not exist" Jun 02 19:05:29.247810 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.247780 2574 scope.go:117] "RemoveContainer" containerID="ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd" Jun 02 19:05:29.247995 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:29.247981 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd\": container with ID starting with ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd not found: ID does not exist" containerID="ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd" Jun 02 19:05:29.248054 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.247997 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd"} err="failed to get container status \"ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd\": rpc error: code = NotFound desc = could not find container \"ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd\": container with ID starting with ccfdf31e69c61e611840c6cd54b90b9c87d3c02b7ca84684efde5de0352fe2dd not found: ID does not exist" Jun 02 19:05:29.248054 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.248008 2574 scope.go:117] "RemoveContainer" containerID="3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb" Jun 02 19:05:29.248170 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:29.248157 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb\": container with ID starting with 3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb not found: ID does not exist" containerID="3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb" Jun 02 19:05:29.248212 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.248173 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb"} err="failed to get container status \"3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb\": rpc error: code = NotFound desc = could not find container \"3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb\": container with ID starting with 3b25178d334eabd54e867f9df2582d2df1930d7f7f904be1d6b11352a97f9fbb not found: ID does not exist" Jun 02 19:05:29.683108 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:29.683083 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" path="/var/lib/kubelet/pods/f0d5f2e4-5750-4463-b72f-dd93bb136eee/volumes" Jun 02 19:05:31.270593 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.270553 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-779c6f5b94-bc5r7" podUID="4dea0451-37c1-46d2-8532-a8ec8427f3c3" containerName="console" containerID="cri-o://cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb" gracePeriod=15 Jun 02 19:05:31.553679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.553659 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-779c6f5b94-bc5r7_4dea0451-37c1-46d2-8532-a8ec8427f3c3/console/0.log" Jun 02 19:05:31.553779 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.553717 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:05:31.664089 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.664059 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-oauth-config\") pod \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " Jun 02 19:05:31.664235 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.664121 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-service-ca\") pod \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " Jun 02 19:05:31.664235 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.664141 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-serving-cert\") pod \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " Jun 02 19:05:31.664235 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.664160 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpqwb\" (UniqueName: \"kubernetes.io/projected/4dea0451-37c1-46d2-8532-a8ec8427f3c3-kube-api-access-kpqwb\") pod \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " Jun 02 19:05:31.664235 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.664178 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-config\") pod \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " Jun 02 19:05:31.664235 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.664195 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-oauth-serving-cert\") pod \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\" (UID: \"4dea0451-37c1-46d2-8532-a8ec8427f3c3\") " Jun 02 19:05:31.664618 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.664580 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-config" (OuterVolumeSpecName: "console-config") pod "4dea0451-37c1-46d2-8532-a8ec8427f3c3" (UID: "4dea0451-37c1-46d2-8532-a8ec8427f3c3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:05:31.664737 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.664631 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-service-ca" (OuterVolumeSpecName: "service-ca") pod "4dea0451-37c1-46d2-8532-a8ec8427f3c3" (UID: "4dea0451-37c1-46d2-8532-a8ec8427f3c3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:05:31.664737 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.664714 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4dea0451-37c1-46d2-8532-a8ec8427f3c3" (UID: "4dea0451-37c1-46d2-8532-a8ec8427f3c3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:05:31.666426 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.666408 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4dea0451-37c1-46d2-8532-a8ec8427f3c3" (UID: "4dea0451-37c1-46d2-8532-a8ec8427f3c3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:31.666726 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.666701 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4dea0451-37c1-46d2-8532-a8ec8427f3c3" (UID: "4dea0451-37c1-46d2-8532-a8ec8427f3c3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:31.666797 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.666721 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dea0451-37c1-46d2-8532-a8ec8427f3c3-kube-api-access-kpqwb" (OuterVolumeSpecName: "kube-api-access-kpqwb") pod "4dea0451-37c1-46d2-8532-a8ec8427f3c3" (UID: "4dea0451-37c1-46d2-8532-a8ec8427f3c3"). InnerVolumeSpecName "kube-api-access-kpqwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:05:31.695167 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.695142 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:05:31.695804 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.695710 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="prometheus" containerID="cri-o://37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65" gracePeriod=600 Jun 02 19:05:31.695804 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.695745 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy" containerID="cri-o://8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911" gracePeriod=600 Jun 02 19:05:31.695804 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.695737 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847" gracePeriod=600 Jun 02 19:05:31.695804 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.695790 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="config-reloader" containerID="cri-o://04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96" gracePeriod=600 Jun 02 19:05:31.696067 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.695746 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy-web" containerID="cri-o://0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984" gracePeriod=600 Jun 02 19:05:31.696067 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.695754 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="thanos-sidecar" containerID="cri-o://832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259" gracePeriod=600 Jun 02 19:05:31.765433 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.765407 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-service-ca\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:31.765433 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.765431 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-serving-cert\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:31.765547 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.765458 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpqwb\" (UniqueName: \"kubernetes.io/projected/4dea0451-37c1-46d2-8532-a8ec8427f3c3-kube-api-access-kpqwb\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:31.765547 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.765468 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-config\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:31.765547 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.765477 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4dea0451-37c1-46d2-8532-a8ec8427f3c3-oauth-serving-cert\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:31.765547 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:31.765485 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4dea0451-37c1-46d2-8532-a8ec8427f3c3-console-oauth-config\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:32.203135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.203109 2574 generic.go:358] "Generic (PLEG): container finished" podID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerID="b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847" exitCode=0 Jun 02 19:05:32.203135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.203131 2574 generic.go:358] "Generic (PLEG): container finished" podID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerID="8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911" exitCode=0 Jun 02 19:05:32.203135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.203137 2574 generic.go:358] "Generic (PLEG): container finished" podID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerID="832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259" exitCode=0 Jun 02 19:05:32.203339 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.203143 2574 generic.go:358] "Generic (PLEG): container finished" podID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerID="04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96" exitCode=0 Jun 02 19:05:32.203339 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.203149 2574 generic.go:358] "Generic (PLEG): container finished" podID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerID="37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65" exitCode=0 Jun 02 19:05:32.203339 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.203183 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerDied","Data":"b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847"} Jun 02 19:05:32.203339 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.203230 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerDied","Data":"8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911"} Jun 02 19:05:32.203339 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.203243 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerDied","Data":"832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259"} Jun 02 19:05:32.203339 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.203252 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerDied","Data":"04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96"} Jun 02 19:05:32.203339 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.203264 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerDied","Data":"37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65"} Jun 02 19:05:32.204318 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.204304 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-779c6f5b94-bc5r7_4dea0451-37c1-46d2-8532-a8ec8427f3c3/console/0.log" Jun 02 19:05:32.204394 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.204334 2574 generic.go:358] "Generic (PLEG): container finished" podID="4dea0451-37c1-46d2-8532-a8ec8427f3c3" containerID="cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb" exitCode=2 Jun 02 19:05:32.204394 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.204358 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779c6f5b94-bc5r7" event={"ID":"4dea0451-37c1-46d2-8532-a8ec8427f3c3","Type":"ContainerDied","Data":"cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb"} Jun 02 19:05:32.204394 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.204388 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779c6f5b94-bc5r7" event={"ID":"4dea0451-37c1-46d2-8532-a8ec8427f3c3","Type":"ContainerDied","Data":"ad6dff27f7387f3a147d56edec18e615f2984752cb0e83c1e87265633292e701"} Jun 02 19:05:32.204486 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.204404 2574 scope.go:117] "RemoveContainer" containerID="cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb" Jun 02 19:05:32.204486 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.204428 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779c6f5b94-bc5r7" Jun 02 19:05:32.211750 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.211733 2574 scope.go:117] "RemoveContainer" containerID="cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb" Jun 02 19:05:32.212003 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:32.211984 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb\": container with ID starting with cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb not found: ID does not exist" containerID="cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb" Jun 02 19:05:32.212054 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.212008 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb"} err="failed to get container status \"cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb\": rpc error: code = NotFound desc = could not find container \"cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb\": container with ID starting with cc563ca438ab66b57c3d237bb7f477a50607a0f048606c97d095b35e420e8afb not found: ID does not exist" Jun 02 19:05:32.219530 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.219509 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-779c6f5b94-bc5r7"] Jun 02 19:05:32.221511 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:32.221489 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-779c6f5b94-bc5r7"] Jun 02 19:05:33.041945 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.041926 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.176395 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176373 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-tls\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176526 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176421 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-metrics-client-ca\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176526 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176439 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-grpc-tls\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176526 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176460 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-tls-assets\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176526 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176481 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-trusted-ca-bundle\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176526 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176500 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176526 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176520 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176535 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-serving-certs-ca-bundle\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176562 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176611 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-kube-rbac-proxy\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176651 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-metrics-client-certs\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176685 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-web-config\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176718 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-rulefiles-0\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176750 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-kubelet-serving-ca-bundle\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176778 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2hp2\" (UniqueName: \"kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-kube-api-access-n2hp2\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.176821 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176804 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-db\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.177261 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176855 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-thanos-prometheus-http-client-file\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.177261 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176890 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config-out\") pod \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\" (UID: \"b0f8ebd4-9251-4b2e-95ae-a54845dab544\") " Jun 02 19:05:33.177261 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.176938 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:05:33.177261 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.177120 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:05:33.177261 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.177178 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.179011 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.177811 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:05:33.179011 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.178758 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:05:33.179011 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.178785 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:05:33.179011 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.178854 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:05:33.179504 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.179478 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:33.179908 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.179877 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:33.180028 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.180005 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:05:33.180444 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.180419 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config" (OuterVolumeSpecName: "config") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:33.180699 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.180675 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config-out" (OuterVolumeSpecName: "config-out") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:05:33.181047 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.181013 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-kube-api-access-n2hp2" (OuterVolumeSpecName: "kube-api-access-n2hp2") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "kube-api-access-n2hp2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:05:33.181431 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.181407 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:33.181527 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.181430 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:33.181527 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.181444 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:33.181527 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.181463 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:33.182538 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.182522 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:33.192445 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.192424 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-web-config" (OuterVolumeSpecName: "web-config") pod "b0f8ebd4-9251-4b2e-95ae-a54845dab544" (UID: "b0f8ebd4-9251-4b2e-95ae-a54845dab544"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:05:33.210468 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.210440 2574 generic.go:358] "Generic (PLEG): container finished" podID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerID="0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984" exitCode=0 Jun 02 19:05:33.210543 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.210479 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerDied","Data":"0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984"} Jun 02 19:05:33.210543 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.210500 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0f8ebd4-9251-4b2e-95ae-a54845dab544","Type":"ContainerDied","Data":"8db49e99dafbf7f2ce8a71132d392838f36ca015adcb266ec964a5307b7d7fee"} Jun 02 19:05:33.210543 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.210514 2574 scope.go:117] "RemoveContainer" containerID="b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847" Jun 02 19:05:33.210543 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.210537 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.220439 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.220422 2574 scope.go:117] "RemoveContainer" containerID="8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911" Jun 02 19:05:33.226517 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.226500 2574 scope.go:117] "RemoveContainer" containerID="0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984" Jun 02 19:05:33.233057 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.233038 2574 scope.go:117] "RemoveContainer" containerID="832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259" Jun 02 19:05:33.233740 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.233719 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:05:33.237095 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.237075 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:05:33.241003 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.240987 2574 scope.go:117] "RemoveContainer" containerID="04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96" Jun 02 19:05:33.247179 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.247163 2574 scope.go:117] "RemoveContainer" containerID="37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65" Jun 02 19:05:33.253533 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.253519 2574 scope.go:117] "RemoveContainer" containerID="35dfd83e329febce11e030416c0bf22fdf1d8a2b3deae9710e66f3824f9a5e32" Jun 02 19:05:33.260130 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.260114 2574 scope.go:117] "RemoveContainer" containerID="b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847" Jun 02 19:05:33.260420 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:33.260402 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847\": container with ID starting with b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847 not found: ID does not exist" containerID="b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847" Jun 02 19:05:33.260492 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.260428 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847"} err="failed to get container status \"b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847\": rpc error: code = NotFound desc = could not find container \"b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847\": container with ID starting with b920f6f5482f91ba4890a2337b57969e1512a1a77e67ebc00df5256002541847 not found: ID does not exist" Jun 02 19:05:33.260492 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.260449 2574 scope.go:117] "RemoveContainer" containerID="8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911" Jun 02 19:05:33.260686 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.260666 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:05:33.260766 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:33.260696 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911\": container with ID starting with 8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911 not found: ID does not exist" containerID="8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911" Jun 02 19:05:33.260766 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.260725 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911"} err="failed to get container status \"8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911\": rpc error: code = NotFound desc = could not find container \"8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911\": container with ID starting with 8751ba689580705fc0e9ebb923df5e977c1cb1ad4b79401225709dbcd8a70911 not found: ID does not exist" Jun 02 19:05:33.260766 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.260748 2574 scope.go:117] "RemoveContainer" containerID="0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984" Jun 02 19:05:33.261019 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:33.261002 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984\": container with ID starting with 0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984 not found: ID does not exist" containerID="0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984" Jun 02 19:05:33.261058 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261031 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984"} err="failed to get container status \"0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984\": rpc error: code = NotFound desc = could not find container \"0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984\": container with ID starting with 0f6a9aba2d976a77b60ec3fc0846e4c4586b1f0e04634596832a91ce53134984 not found: ID does not exist" Jun 02 19:05:33.261058 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261046 2574 scope.go:117] "RemoveContainer" containerID="832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259" Jun 02 19:05:33.261135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261006 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="prometheus" Jun 02 19:05:33.261135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261087 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="prometheus" Jun 02 19:05:33.261135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261104 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="init-config-reloader" Jun 02 19:05:33.261135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261113 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="init-config-reloader" Jun 02 19:05:33.261135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261127 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy-metric" Jun 02 19:05:33.261135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261135 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy-metric" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261149 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy-web" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261158 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy-web" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261167 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="thanos-sidecar" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261175 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="thanos-sidecar" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261184 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dea0451-37c1-46d2-8532-a8ec8427f3c3" containerName="console" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261193 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dea0451-37c1-46d2-8532-a8ec8427f3c3" containerName="console" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261202 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="config-reloader" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261210 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="config-reloader" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261248 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261256 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261263 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261268 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261277 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="prom-label-proxy" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261285 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="prom-label-proxy" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261296 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="alertmanager" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261301 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="alertmanager" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261309 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="config-reloader" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261315 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="config-reloader" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261321 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="init-config-reloader" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261326 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="init-config-reloader" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261331 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy-web" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261343 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy-web" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261350 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy-thanos" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261357 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy-thanos" Jun 02 19:05:33.261396 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:33.261322 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259\": container with ID starting with 832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259 not found: ID does not exist" containerID="832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261428 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="alertmanager" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261423 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259"} err="failed to get container status \"832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259\": rpc error: code = NotFound desc = could not find container \"832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259\": container with ID starting with 832b4f7fa640c26704e2c0c700946ec1d174835b219f9bd03bab5bd6a9071259 not found: ID does not exist" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261437 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261439 2574 scope.go:117] "RemoveContainer" containerID="04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261445 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy-thanos" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261452 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261459 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="config-reloader" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261465 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="4dea0451-37c1-46d2-8532-a8ec8427f3c3" containerName="console" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261474 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="config-reloader" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261483 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy-web" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261491 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="prom-label-proxy" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261500 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="thanos-sidecar" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261509 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="prometheus" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261518 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" containerName="kube-rbac-proxy-web" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261527 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="f0d5f2e4-5750-4463-b72f-dd93bb136eee" containerName="kube-rbac-proxy-metric" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:33.261667 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96\": container with ID starting with 04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96 not found: ID does not exist" containerID="04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261682 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96"} err="failed to get container status \"04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96\": rpc error: code = NotFound desc = could not find container \"04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96\": container with ID starting with 04e7bbe765a5f1016006a9dcfda4e5ee48e860614ec835875591d38ef2a75c96 not found: ID does not exist" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261695 2574 scope.go:117] "RemoveContainer" containerID="37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:33.261908 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65\": container with ID starting with 37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65 not found: ID does not exist" containerID="37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261927 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65"} err="failed to get container status \"37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65\": rpc error: code = NotFound desc = could not find container \"37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65\": container with ID starting with 37b887db0d7208b86f3562159d6b365fabc06e55a531691f9dd465a3a1878c65 not found: ID does not exist" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.261941 2574 scope.go:117] "RemoveContainer" containerID="35dfd83e329febce11e030416c0bf22fdf1d8a2b3deae9710e66f3824f9a5e32" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:05:33.262181 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35dfd83e329febce11e030416c0bf22fdf1d8a2b3deae9710e66f3824f9a5e32\": container with ID starting with 35dfd83e329febce11e030416c0bf22fdf1d8a2b3deae9710e66f3824f9a5e32 not found: ID does not exist" containerID="35dfd83e329febce11e030416c0bf22fdf1d8a2b3deae9710e66f3824f9a5e32" Jun 02 19:05:33.262327 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.262197 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35dfd83e329febce11e030416c0bf22fdf1d8a2b3deae9710e66f3824f9a5e32"} err="failed to get container status \"35dfd83e329febce11e030416c0bf22fdf1d8a2b3deae9710e66f3824f9a5e32\": rpc error: code = NotFound desc = could not find container \"35dfd83e329febce11e030416c0bf22fdf1d8a2b3deae9710e66f3824f9a5e32\": container with ID starting with 35dfd83e329febce11e030416c0bf22fdf1d8a2b3deae9710e66f3824f9a5e32 not found: ID does not exist" Jun 02 19:05:33.265143 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.265129 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.267669 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.267626 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-93fimpkgj6mqn\"" Jun 02 19:05:33.267669 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.267626 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 02 19:05:33.267915 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.267837 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 02 19:05:33.268054 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.268033 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 02 19:05:33.268135 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.268099 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 02 19:05:33.268287 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.268213 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 02 19:05:33.268342 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.268326 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 02 19:05:33.268468 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.268451 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 02 19:05:33.268546 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.268470 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 02 19:05:33.268546 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.268541 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 02 19:05:33.268730 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.268613 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2kvqz\"" Jun 02 19:05:33.268908 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.268870 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 02 19:05:33.271739 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.271723 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 02 19:05:33.274155 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.274135 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 02 19:05:33.277669 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.277648 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:05:33.278030 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278008 2574 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278085 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278035 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config-out\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278085 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278051 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278085 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278064 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-metrics-client-ca\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278085 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278078 2574 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-grpc-tls\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278208 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278091 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-tls-assets\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278208 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278103 2574 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-config\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278208 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278117 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278208 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278130 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278208 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278144 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278208 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278156 2574 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-kube-rbac-proxy\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278208 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278170 2574 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-secret-metrics-client-certs\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278208 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278183 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0f8ebd4-9251-4b2e-95ae-a54845dab544-web-config\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278208 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278195 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278208 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278208 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0f8ebd4-9251-4b2e-95ae-a54845dab544-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278503 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278239 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2hp2\" (UniqueName: \"kubernetes.io/projected/b0f8ebd4-9251-4b2e-95ae-a54845dab544-kube-api-access-n2hp2\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.278503 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.278251 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0f8ebd4-9251-4b2e-95ae-a54845dab544-prometheus-k8s-db\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:05:33.379125 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379103 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379195 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379131 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379195 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379150 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379294 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379194 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvpnq\" (UniqueName: \"kubernetes.io/projected/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-kube-api-access-pvpnq\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379294 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379253 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379294 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379279 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-config\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379384 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379307 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-config-out\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379384 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379325 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379384 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379357 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-web-config\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379384 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379381 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379492 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379403 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379492 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379447 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379492 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379479 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379613 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379501 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379613 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379528 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379613 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379545 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379613 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379563 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.379613 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.379584 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480430 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480372 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-config-out\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480430 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480409 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480430 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480428 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-web-config\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480586 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480460 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480586 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480479 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480586 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480494 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480586 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480519 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480586 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480544 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480586 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480569 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480824 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480592 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480824 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480619 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480824 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480641 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480824 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480679 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480824 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480703 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480824 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480724 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480824 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480739 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pvpnq\" (UniqueName: \"kubernetes.io/projected/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-kube-api-access-pvpnq\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480824 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480770 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.480824 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.480788 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-config\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.483642 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.483402 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-config-out\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.483642 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.483489 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.483642 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.483525 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.483642 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.483588 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.484114 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.483987 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.484361 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.484326 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-web-config\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.484486 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.484461 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.484545 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.484488 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-config\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.484657 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.484590 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.486757 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.484997 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.486757 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.485424 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.486757 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.485663 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.486757 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.486544 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.486757 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.486564 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.490864 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.487009 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.490864 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.489353 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.491984 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.491963 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.492584 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.492565 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvpnq\" (UniqueName: \"kubernetes.io/projected/e3e1ad6e-7c1d-4121-b9e9-1bb659453659-kube-api-access-pvpnq\") pod \"prometheus-k8s-0\" (UID: \"e3e1ad6e-7c1d-4121-b9e9-1bb659453659\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.574978 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.574957 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:33.682973 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.682943 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dea0451-37c1-46d2-8532-a8ec8427f3c3" path="/var/lib/kubelet/pods/4dea0451-37c1-46d2-8532-a8ec8427f3c3/volumes" Jun 02 19:05:33.683377 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.683361 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0f8ebd4-9251-4b2e-95ae-a54845dab544" path="/var/lib/kubelet/pods/b0f8ebd4-9251-4b2e-95ae-a54845dab544/volumes" Jun 02 19:05:33.709650 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:33.709566 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 19:05:33.712354 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:05:33.712329 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e1ad6e_7c1d_4121_b9e9_1bb659453659.slice/crio-c580526be952eacafad84c6cbae33d7240ceee35d5185bfb35e47709a515c980 WatchSource:0}: Error finding container c580526be952eacafad84c6cbae33d7240ceee35d5185bfb35e47709a515c980: Status 404 returned error can't find the container with id c580526be952eacafad84c6cbae33d7240ceee35d5185bfb35e47709a515c980 Jun 02 19:05:34.216182 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:34.216154 2574 generic.go:358] "Generic (PLEG): container finished" podID="e3e1ad6e-7c1d-4121-b9e9-1bb659453659" containerID="811215c5f717624aac293f85ee279f018ad4668e7f35f06ebaaf715dd956dd26" exitCode=0 Jun 02 19:05:34.216522 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:34.216236 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3e1ad6e-7c1d-4121-b9e9-1bb659453659","Type":"ContainerDied","Data":"811215c5f717624aac293f85ee279f018ad4668e7f35f06ebaaf715dd956dd26"} Jun 02 19:05:34.216522 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:34.216259 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3e1ad6e-7c1d-4121-b9e9-1bb659453659","Type":"ContainerStarted","Data":"c580526be952eacafad84c6cbae33d7240ceee35d5185bfb35e47709a515c980"} Jun 02 19:05:35.223198 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:35.223164 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3e1ad6e-7c1d-4121-b9e9-1bb659453659","Type":"ContainerStarted","Data":"2d559e22440c24eec662edbb3289a1b08c3e7eebddac1d3f6dae46918f819ef0"} Jun 02 19:05:35.223198 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:35.223202 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3e1ad6e-7c1d-4121-b9e9-1bb659453659","Type":"ContainerStarted","Data":"125a49af29f0e1b6e696127926cce4f8b7b5ac030594bbcb29782736be7458bf"} Jun 02 19:05:35.223604 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:35.223213 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3e1ad6e-7c1d-4121-b9e9-1bb659453659","Type":"ContainerStarted","Data":"03ee25d2cae219dd8afa4ab42f0e39a58b7252de381f6163193adc284aa83e16"} Jun 02 19:05:35.223604 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:35.223239 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3e1ad6e-7c1d-4121-b9e9-1bb659453659","Type":"ContainerStarted","Data":"8060c88eba67a399e74e2003658b319e29e8cdd0e16917eda36a36e79e6eb533"} Jun 02 19:05:35.223604 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:35.223248 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3e1ad6e-7c1d-4121-b9e9-1bb659453659","Type":"ContainerStarted","Data":"95ac6030778a29e37df8ee539fb40fb921d1a77926a27c58b3ccb0e50dc37e24"} Jun 02 19:05:35.223604 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:35.223257 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3e1ad6e-7c1d-4121-b9e9-1bb659453659","Type":"ContainerStarted","Data":"52076301c3c84537d000781e14daa222399676b89802a62c91425d3a759cd556"} Jun 02 19:05:35.248157 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:35.248110 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.2480973730000002 podStartE2EDuration="2.248097373s" podCreationTimestamp="2026-06-02 19:05:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:05:35.245887741 +0000 UTC m=+98.063857466" watchObservedRunningTime="2026-06-02 19:05:35.248097373 +0000 UTC m=+98.066067096" Jun 02 19:05:38.575556 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:38.575524 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:05:40.121359 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:40.121329 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xc29k" Jun 02 19:05:41.291666 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:05:41.291634 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8567df9dc9-k5f7r"] Jun 02 19:06:06.311535 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.311473 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-8567df9dc9-k5f7r" podUID="03f3828b-2b66-429f-a69c-753c863bd02f" containerName="console" containerID="cri-o://8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a" gracePeriod=15 Jun 02 19:06:06.558142 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.558113 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8567df9dc9-k5f7r_03f3828b-2b66-429f-a69c-753c863bd02f/console/0.log" Jun 02 19:06:06.558262 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.558184 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:06:06.629786 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.629709 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-console-config\") pod \"03f3828b-2b66-429f-a69c-753c863bd02f\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " Jun 02 19:06:06.629786 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.629750 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-serving-cert\") pod \"03f3828b-2b66-429f-a69c-753c863bd02f\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " Jun 02 19:06:06.629786 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.629781 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-oauth-config\") pod \"03f3828b-2b66-429f-a69c-753c863bd02f\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " Jun 02 19:06:06.630044 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.629815 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw74b\" (UniqueName: \"kubernetes.io/projected/03f3828b-2b66-429f-a69c-753c863bd02f-kube-api-access-mw74b\") pod \"03f3828b-2b66-429f-a69c-753c863bd02f\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " Jun 02 19:06:06.630044 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.629839 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-service-ca\") pod \"03f3828b-2b66-429f-a69c-753c863bd02f\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " Jun 02 19:06:06.630044 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.629864 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-trusted-ca-bundle\") pod \"03f3828b-2b66-429f-a69c-753c863bd02f\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " Jun 02 19:06:06.630044 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.629983 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-oauth-serving-cert\") pod \"03f3828b-2b66-429f-a69c-753c863bd02f\" (UID: \"03f3828b-2b66-429f-a69c-753c863bd02f\") " Jun 02 19:06:06.630252 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.630099 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-console-config" (OuterVolumeSpecName: "console-config") pod "03f3828b-2b66-429f-a69c-753c863bd02f" (UID: "03f3828b-2b66-429f-a69c-753c863bd02f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:06:06.630385 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.630364 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-service-ca" (OuterVolumeSpecName: "service-ca") pod "03f3828b-2b66-429f-a69c-753c863bd02f" (UID: "03f3828b-2b66-429f-a69c-753c863bd02f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:06:06.630438 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.630384 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "03f3828b-2b66-429f-a69c-753c863bd02f" (UID: "03f3828b-2b66-429f-a69c-753c863bd02f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:06:06.630475 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.630445 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "03f3828b-2b66-429f-a69c-753c863bd02f" (UID: "03f3828b-2b66-429f-a69c-753c863bd02f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 19:06:06.632106 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.632084 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f3828b-2b66-429f-a69c-753c863bd02f-kube-api-access-mw74b" (OuterVolumeSpecName: "kube-api-access-mw74b") pod "03f3828b-2b66-429f-a69c-753c863bd02f" (UID: "03f3828b-2b66-429f-a69c-753c863bd02f"). InnerVolumeSpecName "kube-api-access-mw74b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:06:06.632397 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.632363 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "03f3828b-2b66-429f-a69c-753c863bd02f" (UID: "03f3828b-2b66-429f-a69c-753c863bd02f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:06:06.632397 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.632382 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "03f3828b-2b66-429f-a69c-753c863bd02f" (UID: "03f3828b-2b66-429f-a69c-753c863bd02f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:06:06.731294 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.731131 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-oauth-serving-cert\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:06:06.731294 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.731169 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-console-config\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:06:06.731294 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.731186 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-serving-cert\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:06:06.731294 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.731200 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03f3828b-2b66-429f-a69c-753c863bd02f-console-oauth-config\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:06:06.731294 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.731237 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw74b\" (UniqueName: \"kubernetes.io/projected/03f3828b-2b66-429f-a69c-753c863bd02f-kube-api-access-mw74b\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:06:06.731294 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.731252 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-service-ca\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:06:06.731294 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:06.731266 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f3828b-2b66-429f-a69c-753c863bd02f-trusted-ca-bundle\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:06:07.323058 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:07.323028 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8567df9dc9-k5f7r_03f3828b-2b66-429f-a69c-753c863bd02f/console/0.log" Jun 02 19:06:07.323504 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:07.323073 2574 generic.go:358] "Generic (PLEG): container finished" podID="03f3828b-2b66-429f-a69c-753c863bd02f" containerID="8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a" exitCode=2 Jun 02 19:06:07.323504 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:07.323162 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8567df9dc9-k5f7r" Jun 02 19:06:07.323504 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:07.323164 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8567df9dc9-k5f7r" event={"ID":"03f3828b-2b66-429f-a69c-753c863bd02f","Type":"ContainerDied","Data":"8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a"} Jun 02 19:06:07.323504 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:07.323202 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8567df9dc9-k5f7r" event={"ID":"03f3828b-2b66-429f-a69c-753c863bd02f","Type":"ContainerDied","Data":"7d55815bbcc14a71b15c9c9ffdad2e9056182b38c33ec4df47457bd4d1df122a"} Jun 02 19:06:07.323504 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:07.323231 2574 scope.go:117] "RemoveContainer" containerID="8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a" Jun 02 19:06:07.333232 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:07.333200 2574 scope.go:117] "RemoveContainer" containerID="8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a" Jun 02 19:06:07.333480 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:06:07.333462 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a\": container with ID starting with 8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a not found: ID does not exist" containerID="8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a" Jun 02 19:06:07.333539 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:07.333486 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a"} err="failed to get container status \"8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a\": rpc error: code = NotFound desc = could not find container \"8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a\": container with ID starting with 8f898baec57a94b507f18ddd8673fa7f8cdefbc574c3da452a58a5fc396d968a not found: ID does not exist" Jun 02 19:06:07.349448 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:07.349418 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8567df9dc9-k5f7r"] Jun 02 19:06:07.351411 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:07.351388 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8567df9dc9-k5f7r"] Jun 02 19:06:07.682347 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:07.682321 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03f3828b-2b66-429f-a69c-753c863bd02f" path="/var/lib/kubelet/pods/03f3828b-2b66-429f-a69c-753c863bd02f/volumes" Jun 02 19:06:33.575972 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:33.575934 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:06:33.590974 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:33.590950 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:06:34.419120 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:06:34.419090 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 19:07:39.688021 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.687936 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf"] Jun 02 19:07:39.688535 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.688406 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f3828b-2b66-429f-a69c-753c863bd02f" containerName="console" Jun 02 19:07:39.688535 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.688423 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f3828b-2b66-429f-a69c-753c863bd02f" containerName="console" Jun 02 19:07:39.688535 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.688498 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="03f3828b-2b66-429f-a69c-753c863bd02f" containerName="console" Jun 02 19:07:39.690490 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.690464 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.692648 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.692626 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 19:07:39.692774 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.692758 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 19:07:39.692900 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.692882 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 19:07:39.692958 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.692946 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wnhl7\"" Jun 02 19:07:39.692994 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.692949 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 19:07:39.702977 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.702955 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf"] Jun 02 19:07:39.786916 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.786880 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhc26\" (UniqueName: \"kubernetes.io/projected/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-kube-api-access-nhc26\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.787095 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.786937 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-manifests\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.787095 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.787001 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.787095 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.787031 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-tmp\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.787095 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.787054 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-webhook-cert\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.887834 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.887805 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-manifests\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.887957 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.887861 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.887957 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.887888 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-tmp\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.887957 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.887907 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-webhook-cert\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.887957 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.887948 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nhc26\" (UniqueName: \"kubernetes.io/projected/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-kube-api-access-nhc26\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.888284 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.888263 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-manifests\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.890166 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.890145 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-tmp\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.890397 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.890377 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.890532 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.890515 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-webhook-cert\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:39.895741 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:39.895721 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhc26\" (UniqueName: \"kubernetes.io/projected/f6aca53f-2c1f-41dc-9fd2-879c1a315b22-kube-api-access-nhc26\") pod \"opendatahub-operator-controller-manager-69b56f574b-mrctf\" (UID: \"f6aca53f-2c1f-41dc-9fd2-879c1a315b22\") " pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:40.000419 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:40.000324 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:40.131539 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:40.131506 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf"] Jun 02 19:07:40.134872 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:07:40.134837 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6aca53f_2c1f_41dc_9fd2_879c1a315b22.slice/crio-d1573bde9561ed02bfd3943e936f0a96d7e793aeb76493226baf6d95fd04e0e1 WatchSource:0}: Error finding container d1573bde9561ed02bfd3943e936f0a96d7e793aeb76493226baf6d95fd04e0e1: Status 404 returned error can't find the container with id d1573bde9561ed02bfd3943e936f0a96d7e793aeb76493226baf6d95fd04e0e1 Jun 02 19:07:40.587744 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:40.587705 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" event={"ID":"f6aca53f-2c1f-41dc-9fd2-879c1a315b22","Type":"ContainerStarted","Data":"d1573bde9561ed02bfd3943e936f0a96d7e793aeb76493226baf6d95fd04e0e1"} Jun 02 19:07:43.597846 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:43.597811 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" event={"ID":"f6aca53f-2c1f-41dc-9fd2-879c1a315b22","Type":"ContainerStarted","Data":"736dd67c47d5564910438b8f7343edc0bfa997b08ecbb02ea622b0c9761817eb"} Jun 02 19:07:44.601843 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:44.601813 2574 generic.go:358] "Generic (PLEG): container finished" podID="f6aca53f-2c1f-41dc-9fd2-879c1a315b22" containerID="736dd67c47d5564910438b8f7343edc0bfa997b08ecbb02ea622b0c9761817eb" exitCode=0 Jun 02 19:07:44.602152 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:44.601872 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" event={"ID":"f6aca53f-2c1f-41dc-9fd2-879c1a315b22","Type":"ContainerDied","Data":"736dd67c47d5564910438b8f7343edc0bfa997b08ecbb02ea622b0c9761817eb"} Jun 02 19:07:45.606394 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:45.606363 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" event={"ID":"f6aca53f-2c1f-41dc-9fd2-879c1a315b22","Type":"ContainerStarted","Data":"388bf0ad8094bbecf2280eb16d55c2278f387e9eae9bacbcb3839b2b3bc862d7"} Jun 02 19:07:45.606843 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:45.606466 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:45.626793 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:45.626747 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" podStartSLOduration=1.838674865 podStartE2EDuration="6.626733706s" podCreationTimestamp="2026-06-02 19:07:39 +0000 UTC" firstStartedPulling="2026-06-02 19:07:40.136574997 +0000 UTC m=+222.954544702" lastFinishedPulling="2026-06-02 19:07:44.924633835 +0000 UTC m=+227.742603543" observedRunningTime="2026-06-02 19:07:45.624771444 +0000 UTC m=+228.442741168" watchObservedRunningTime="2026-06-02 19:07:45.626733706 +0000 UTC m=+228.444703430" Jun 02 19:07:56.611590 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:56.611559 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-69b56f574b-mrctf" Jun 02 19:07:57.091108 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.091075 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr"] Jun 02 19:07:57.103203 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.103176 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr"] Jun 02 19:07:57.103349 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.103315 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.106303 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.106277 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 02 19:07:57.106456 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.106436 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 02 19:07:57.106537 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.106523 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 02 19:07:57.106944 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.106924 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4xskr\"" Jun 02 19:07:57.111085 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.111060 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 02 19:07:57.111176 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.111132 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 02 19:07:57.232670 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.232635 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b04d2724-3ba4-45bf-9784-daa2858ce165-metrics-cert\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.232835 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.232674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kbj9\" (UniqueName: \"kubernetes.io/projected/b04d2724-3ba4-45bf-9784-daa2858ce165-kube-api-access-9kbj9\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.232835 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.232701 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b04d2724-3ba4-45bf-9784-daa2858ce165-cert\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.232835 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.232803 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b04d2724-3ba4-45bf-9784-daa2858ce165-manager-config\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.334085 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.334045 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b04d2724-3ba4-45bf-9784-daa2858ce165-metrics-cert\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.334301 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.334095 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9kbj9\" (UniqueName: \"kubernetes.io/projected/b04d2724-3ba4-45bf-9784-daa2858ce165-kube-api-access-9kbj9\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.334301 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.334129 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b04d2724-3ba4-45bf-9784-daa2858ce165-cert\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.334301 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.334187 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b04d2724-3ba4-45bf-9784-daa2858ce165-manager-config\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.334840 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.334815 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b04d2724-3ba4-45bf-9784-daa2858ce165-manager-config\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.336501 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.336483 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b04d2724-3ba4-45bf-9784-daa2858ce165-cert\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.336584 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.336568 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b04d2724-3ba4-45bf-9784-daa2858ce165-metrics-cert\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.346475 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.346415 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kbj9\" (UniqueName: \"kubernetes.io/projected/b04d2724-3ba4-45bf-9784-daa2858ce165-kube-api-access-9kbj9\") pod \"lws-controller-manager-6cd7cc9b94-lnxbr\" (UID: \"b04d2724-3ba4-45bf-9784-daa2858ce165\") " pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.413393 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.413367 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:07:57.537465 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.537436 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr"] Jun 02 19:07:57.540339 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:07:57.540315 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb04d2724_3ba4_45bf_9784_daa2858ce165.slice/crio-f40c03b7ef2e9e4b97bf91390123bad7fe01b94151e8a9637a9b967a8388baff WatchSource:0}: Error finding container f40c03b7ef2e9e4b97bf91390123bad7fe01b94151e8a9637a9b967a8388baff: Status 404 returned error can't find the container with id f40c03b7ef2e9e4b97bf91390123bad7fe01b94151e8a9637a9b967a8388baff Jun 02 19:07:57.640588 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:57.640526 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" event={"ID":"b04d2724-3ba4-45bf-9784-daa2858ce165","Type":"ContainerStarted","Data":"f40c03b7ef2e9e4b97bf91390123bad7fe01b94151e8a9637a9b967a8388baff"} Jun 02 19:07:59.107145 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.107104 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr"] Jun 02 19:07:59.116120 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.116098 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" Jun 02 19:07:59.120607 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.120581 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 02 19:07:59.121076 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.121058 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fg6w8\"" Jun 02 19:07:59.121373 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.121347 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 02 19:07:59.121473 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.121455 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 19:07:59.121948 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.121931 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 19:07:59.141526 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.141454 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr"] Jun 02 19:07:59.250983 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.250940 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9505c068-2097-4ab0-98da-648bcdca1989-tls-certs\") pod \"kube-auth-proxy-6f9d7d5cb6-7qqfr\" (UID: \"9505c068-2097-4ab0-98da-648bcdca1989\") " pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" Jun 02 19:07:59.251171 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.251152 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9505c068-2097-4ab0-98da-648bcdca1989-tmp\") pod \"kube-auth-proxy-6f9d7d5cb6-7qqfr\" (UID: \"9505c068-2097-4ab0-98da-648bcdca1989\") " pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" Jun 02 19:07:59.251293 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.251273 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4d7n\" (UniqueName: \"kubernetes.io/projected/9505c068-2097-4ab0-98da-648bcdca1989-kube-api-access-f4d7n\") pod \"kube-auth-proxy-6f9d7d5cb6-7qqfr\" (UID: \"9505c068-2097-4ab0-98da-648bcdca1989\") " pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" Jun 02 19:07:59.352167 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.352128 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9505c068-2097-4ab0-98da-648bcdca1989-tmp\") pod \"kube-auth-proxy-6f9d7d5cb6-7qqfr\" (UID: \"9505c068-2097-4ab0-98da-648bcdca1989\") " pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" Jun 02 19:07:59.352363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.352196 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f4d7n\" (UniqueName: \"kubernetes.io/projected/9505c068-2097-4ab0-98da-648bcdca1989-kube-api-access-f4d7n\") pod \"kube-auth-proxy-6f9d7d5cb6-7qqfr\" (UID: \"9505c068-2097-4ab0-98da-648bcdca1989\") " pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" Jun 02 19:07:59.352363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.352274 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9505c068-2097-4ab0-98da-648bcdca1989-tls-certs\") pod \"kube-auth-proxy-6f9d7d5cb6-7qqfr\" (UID: \"9505c068-2097-4ab0-98da-648bcdca1989\") " pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" Jun 02 19:07:59.354757 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.354729 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9505c068-2097-4ab0-98da-648bcdca1989-tmp\") pod \"kube-auth-proxy-6f9d7d5cb6-7qqfr\" (UID: \"9505c068-2097-4ab0-98da-648bcdca1989\") " pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" Jun 02 19:07:59.355054 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.355033 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9505c068-2097-4ab0-98da-648bcdca1989-tls-certs\") pod \"kube-auth-proxy-6f9d7d5cb6-7qqfr\" (UID: \"9505c068-2097-4ab0-98da-648bcdca1989\") " pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" Jun 02 19:07:59.366967 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.366876 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4d7n\" (UniqueName: \"kubernetes.io/projected/9505c068-2097-4ab0-98da-648bcdca1989-kube-api-access-f4d7n\") pod \"kube-auth-proxy-6f9d7d5cb6-7qqfr\" (UID: \"9505c068-2097-4ab0-98da-648bcdca1989\") " pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" Jun 02 19:07:59.429330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:07:59.429296 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" Jun 02 19:08:00.064890 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:00.064862 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr"] Jun 02 19:08:00.067020 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:08:00.066997 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9505c068_2097_4ab0_98da_648bcdca1989.slice/crio-492d112267f8650c57dafb525aa318842b56b2bf11bd6ce6dfc4c141dbf55930 WatchSource:0}: Error finding container 492d112267f8650c57dafb525aa318842b56b2bf11bd6ce6dfc4c141dbf55930: Status 404 returned error can't find the container with id 492d112267f8650c57dafb525aa318842b56b2bf11bd6ce6dfc4c141dbf55930 Jun 02 19:08:00.652460 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:00.652403 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" event={"ID":"b04d2724-3ba4-45bf-9784-daa2858ce165","Type":"ContainerStarted","Data":"44bc2a84bcc22ad9b31651645ec1a862ca76753427645d2cbfc122d882a1f485"} Jun 02 19:08:00.652974 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:00.652514 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:08:00.654406 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:00.654377 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" event={"ID":"9505c068-2097-4ab0-98da-648bcdca1989","Type":"ContainerStarted","Data":"492d112267f8650c57dafb525aa318842b56b2bf11bd6ce6dfc4c141dbf55930"} Jun 02 19:08:00.669742 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:00.669690 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" podStartSLOduration=1.2149617560000001 podStartE2EDuration="3.669674396s" podCreationTimestamp="2026-06-02 19:07:57 +0000 UTC" firstStartedPulling="2026-06-02 19:07:57.544599391 +0000 UTC m=+240.362569094" lastFinishedPulling="2026-06-02 19:07:59.999312019 +0000 UTC m=+242.817281734" observedRunningTime="2026-06-02 19:08:00.668035379 +0000 UTC m=+243.486005106" watchObservedRunningTime="2026-06-02 19:08:00.669674396 +0000 UTC m=+243.487644121" Jun 02 19:08:02.662620 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:02.662586 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" event={"ID":"9505c068-2097-4ab0-98da-648bcdca1989","Type":"ContainerStarted","Data":"d165fb83a78ebd67a3987e2c39dd3083eb2a23f2f18cb63ab70c096ccd513373"} Jun 02 19:08:02.679879 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:02.679832 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6f9d7d5cb6-7qqfr" podStartSLOduration=1.236361736 podStartE2EDuration="3.679817031s" podCreationTimestamp="2026-06-02 19:07:59 +0000 UTC" firstStartedPulling="2026-06-02 19:08:00.068834846 +0000 UTC m=+242.886804562" lastFinishedPulling="2026-06-02 19:08:02.512290153 +0000 UTC m=+245.330259857" observedRunningTime="2026-06-02 19:08:02.678163182 +0000 UTC m=+245.496132932" watchObservedRunningTime="2026-06-02 19:08:02.679817031 +0000 UTC m=+245.497786755" Jun 02 19:08:11.661305 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:11.661206 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6cd7cc9b94-lnxbr" Jun 02 19:08:29.320240 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.320190 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-5d857d74c6-vg2b7"] Jun 02 19:08:29.323490 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.323470 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-5d857d74c6-vg2b7" Jun 02 19:08:29.326022 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.325998 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 02 19:08:29.326130 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.326003 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vfkhm\"" Jun 02 19:08:29.331883 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.331860 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-5d857d74c6-vg2b7"] Jun 02 19:08:29.407646 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.407620 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/362fab38-9d9f-4d44-b8cd-b728ec0359ec-redirect-config\") pod \"dashboard-redirect-5d857d74c6-vg2b7\" (UID: \"362fab38-9d9f-4d44-b8cd-b728ec0359ec\") " pod="opendatahub/dashboard-redirect-5d857d74c6-vg2b7" Jun 02 19:08:29.407760 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.407654 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fzmv\" (UniqueName: \"kubernetes.io/projected/362fab38-9d9f-4d44-b8cd-b728ec0359ec-kube-api-access-7fzmv\") pod \"dashboard-redirect-5d857d74c6-vg2b7\" (UID: \"362fab38-9d9f-4d44-b8cd-b728ec0359ec\") " pod="opendatahub/dashboard-redirect-5d857d74c6-vg2b7" Jun 02 19:08:29.508665 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.508644 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/362fab38-9d9f-4d44-b8cd-b728ec0359ec-redirect-config\") pod \"dashboard-redirect-5d857d74c6-vg2b7\" (UID: \"362fab38-9d9f-4d44-b8cd-b728ec0359ec\") " pod="opendatahub/dashboard-redirect-5d857d74c6-vg2b7" Jun 02 19:08:29.508773 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.508676 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7fzmv\" (UniqueName: \"kubernetes.io/projected/362fab38-9d9f-4d44-b8cd-b728ec0359ec-kube-api-access-7fzmv\") pod \"dashboard-redirect-5d857d74c6-vg2b7\" (UID: \"362fab38-9d9f-4d44-b8cd-b728ec0359ec\") " pod="opendatahub/dashboard-redirect-5d857d74c6-vg2b7" Jun 02 19:08:29.509319 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.509301 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/362fab38-9d9f-4d44-b8cd-b728ec0359ec-redirect-config\") pod \"dashboard-redirect-5d857d74c6-vg2b7\" (UID: \"362fab38-9d9f-4d44-b8cd-b728ec0359ec\") " pod="opendatahub/dashboard-redirect-5d857d74c6-vg2b7" Jun 02 19:08:29.516444 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.516424 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fzmv\" (UniqueName: \"kubernetes.io/projected/362fab38-9d9f-4d44-b8cd-b728ec0359ec-kube-api-access-7fzmv\") pod \"dashboard-redirect-5d857d74c6-vg2b7\" (UID: \"362fab38-9d9f-4d44-b8cd-b728ec0359ec\") " pod="opendatahub/dashboard-redirect-5d857d74c6-vg2b7" Jun 02 19:08:29.636149 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.636088 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-5d857d74c6-vg2b7" Jun 02 19:08:29.759347 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:29.759320 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-5d857d74c6-vg2b7"] Jun 02 19:08:29.761838 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:08:29.761807 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod362fab38_9d9f_4d44_b8cd_b728ec0359ec.slice/crio-1db5c1235b9e028d728c2edc927025943bb3145823f67d29aa3783dc300bbcaa WatchSource:0}: Error finding container 1db5c1235b9e028d728c2edc927025943bb3145823f67d29aa3783dc300bbcaa: Status 404 returned error can't find the container with id 1db5c1235b9e028d728c2edc927025943bb3145823f67d29aa3783dc300bbcaa Jun 02 19:08:30.746653 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:30.746613 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-5d857d74c6-vg2b7" event={"ID":"362fab38-9d9f-4d44-b8cd-b728ec0359ec","Type":"ContainerStarted","Data":"1db5c1235b9e028d728c2edc927025943bb3145823f67d29aa3783dc300bbcaa"} Jun 02 19:08:32.760127 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:32.760093 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-5d857d74c6-vg2b7" event={"ID":"362fab38-9d9f-4d44-b8cd-b728ec0359ec","Type":"ContainerStarted","Data":"55f31ca400c6ba75441d36cb1e1d8bfb8901eeb629ef6b93d7036f619248b703"} Jun 02 19:08:32.776575 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:32.776524 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-5d857d74c6-vg2b7" podStartSLOduration=1.81390793 podStartE2EDuration="3.776511309s" podCreationTimestamp="2026-06-02 19:08:29 +0000 UTC" firstStartedPulling="2026-06-02 19:08:29.762907461 +0000 UTC m=+272.580877163" lastFinishedPulling="2026-06-02 19:08:31.725510828 +0000 UTC m=+274.543480542" observedRunningTime="2026-06-02 19:08:32.775393231 +0000 UTC m=+275.593362954" watchObservedRunningTime="2026-06-02 19:08:32.776511309 +0000 UTC m=+275.594481032" Jun 02 19:08:57.577307 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:08:57.577283 2574 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 19:09:35.505628 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.505591 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp"] Jun 02 19:09:35.507718 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.507703 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:35.510024 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.509996 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 19:09:35.510160 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.510030 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 19:09:35.510303 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.510287 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q6wps\"" Jun 02 19:09:35.518515 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.518493 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp"] Jun 02 19:09:35.554304 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.554277 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d294eca9-a053-4100-a22e-67329d36c362-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-bcqfp\" (UID: \"d294eca9-a053-4100-a22e-67329d36c362\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:35.554454 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.554317 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td5hj\" (UniqueName: \"kubernetes.io/projected/d294eca9-a053-4100-a22e-67329d36c362-kube-api-access-td5hj\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-bcqfp\" (UID: \"d294eca9-a053-4100-a22e-67329d36c362\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:35.655304 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.655269 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d294eca9-a053-4100-a22e-67329d36c362-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-bcqfp\" (UID: \"d294eca9-a053-4100-a22e-67329d36c362\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:35.655455 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.655312 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-td5hj\" (UniqueName: \"kubernetes.io/projected/d294eca9-a053-4100-a22e-67329d36c362-kube-api-access-td5hj\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-bcqfp\" (UID: \"d294eca9-a053-4100-a22e-67329d36c362\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:35.655724 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.655702 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d294eca9-a053-4100-a22e-67329d36c362-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-bcqfp\" (UID: \"d294eca9-a053-4100-a22e-67329d36c362\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:35.663559 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.663536 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-td5hj\" (UniqueName: \"kubernetes.io/projected/d294eca9-a053-4100-a22e-67329d36c362-kube-api-access-td5hj\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-bcqfp\" (UID: \"d294eca9-a053-4100-a22e-67329d36c362\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:35.818483 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.818391 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:35.952841 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.952815 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp"] Jun 02 19:09:35.955202 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:09:35.955168 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd294eca9_a053_4100_a22e_67329d36c362.slice/crio-376972d43b6ac8770f81e2b64e96677db264c0c58d124a986bb79b811a3e9421 WatchSource:0}: Error finding container 376972d43b6ac8770f81e2b64e96677db264c0c58d124a986bb79b811a3e9421: Status 404 returned error can't find the container with id 376972d43b6ac8770f81e2b64e96677db264c0c58d124a986bb79b811a3e9421 Jun 02 19:09:35.957423 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:35.957403 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 19:09:36.963320 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:36.963246 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" event={"ID":"d294eca9-a053-4100-a22e-67329d36c362","Type":"ContainerStarted","Data":"376972d43b6ac8770f81e2b64e96677db264c0c58d124a986bb79b811a3e9421"} Jun 02 19:09:41.981270 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:41.981234 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" event={"ID":"d294eca9-a053-4100-a22e-67329d36c362","Type":"ContainerStarted","Data":"2177afe4a2e6dda90f983b0a6f58166af6f4fa110c210a54c01f702754528b32"} Jun 02 19:09:41.981661 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:41.981370 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:42.002021 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:42.001915 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" podStartSLOduration=1.217420941 podStartE2EDuration="7.001901682s" podCreationTimestamp="2026-06-02 19:09:35 +0000 UTC" firstStartedPulling="2026-06-02 19:09:35.957527947 +0000 UTC m=+338.775497649" lastFinishedPulling="2026-06-02 19:09:41.742008684 +0000 UTC m=+344.559978390" observedRunningTime="2026-06-02 19:09:41.999694595 +0000 UTC m=+344.817664331" watchObservedRunningTime="2026-06-02 19:09:42.001901682 +0000 UTC m=+344.819871405" Jun 02 19:09:44.588039 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.588008 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w"] Jun 02 19:09:44.590206 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.590190 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" Jun 02 19:09:44.592854 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.592834 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-k9jfh\"" Jun 02 19:09:44.592965 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.592865 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 02 19:09:44.593239 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.593209 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 02 19:09:44.601035 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.601012 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w"] Jun 02 19:09:44.738818 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.738784 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxrc8\" (UniqueName: \"kubernetes.io/projected/d13a3fb7-eb6a-4e14-b984-612e04083bde-kube-api-access-wxrc8\") pod \"kuadrant-console-plugin-6cb54b5c86-jrd8w\" (UID: \"d13a3fb7-eb6a-4e14-b984-612e04083bde\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" Jun 02 19:09:44.738992 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.738837 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d13a3fb7-eb6a-4e14-b984-612e04083bde-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-jrd8w\" (UID: \"d13a3fb7-eb6a-4e14-b984-612e04083bde\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" Jun 02 19:09:44.738992 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.738909 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d13a3fb7-eb6a-4e14-b984-612e04083bde-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-jrd8w\" (UID: \"d13a3fb7-eb6a-4e14-b984-612e04083bde\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" Jun 02 19:09:44.840032 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.839936 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxrc8\" (UniqueName: \"kubernetes.io/projected/d13a3fb7-eb6a-4e14-b984-612e04083bde-kube-api-access-wxrc8\") pod \"kuadrant-console-plugin-6cb54b5c86-jrd8w\" (UID: \"d13a3fb7-eb6a-4e14-b984-612e04083bde\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" Jun 02 19:09:44.840032 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.839994 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d13a3fb7-eb6a-4e14-b984-612e04083bde-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-jrd8w\" (UID: \"d13a3fb7-eb6a-4e14-b984-612e04083bde\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" Jun 02 19:09:44.840032 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.840024 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d13a3fb7-eb6a-4e14-b984-612e04083bde-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-jrd8w\" (UID: \"d13a3fb7-eb6a-4e14-b984-612e04083bde\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" Jun 02 19:09:44.840653 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.840629 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d13a3fb7-eb6a-4e14-b984-612e04083bde-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-jrd8w\" (UID: \"d13a3fb7-eb6a-4e14-b984-612e04083bde\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" Jun 02 19:09:44.842463 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.842444 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d13a3fb7-eb6a-4e14-b984-612e04083bde-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-jrd8w\" (UID: \"d13a3fb7-eb6a-4e14-b984-612e04083bde\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" Jun 02 19:09:44.850264 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.850241 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxrc8\" (UniqueName: \"kubernetes.io/projected/d13a3fb7-eb6a-4e14-b984-612e04083bde-kube-api-access-wxrc8\") pod \"kuadrant-console-plugin-6cb54b5c86-jrd8w\" (UID: \"d13a3fb7-eb6a-4e14-b984-612e04083bde\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" Jun 02 19:09:44.907229 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:44.907197 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" Jun 02 19:09:45.039311 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:45.039285 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w"] Jun 02 19:09:45.041394 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:09:45.041361 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd13a3fb7_eb6a_4e14_b984_612e04083bde.slice/crio-d6ad04c621b46f01aca803ea6ed115dbdbefb2ceed16137cd03bcfcdd3f3c664 WatchSource:0}: Error finding container d6ad04c621b46f01aca803ea6ed115dbdbefb2ceed16137cd03bcfcdd3f3c664: Status 404 returned error can't find the container with id d6ad04c621b46f01aca803ea6ed115dbdbefb2ceed16137cd03bcfcdd3f3c664 Jun 02 19:09:45.994934 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:45.994891 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" event={"ID":"d13a3fb7-eb6a-4e14-b984-612e04083bde","Type":"ContainerStarted","Data":"d6ad04c621b46f01aca803ea6ed115dbdbefb2ceed16137cd03bcfcdd3f3c664"} Jun 02 19:09:52.986296 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:52.986266 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:54.777782 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.777739 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp"] Jun 02 19:09:54.778234 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.777965 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" podUID="d294eca9-a053-4100-a22e-67329d36c362" containerName="manager" containerID="cri-o://2177afe4a2e6dda90f983b0a6f58166af6f4fa110c210a54c01f702754528b32" gracePeriod=2 Jun 02 19:09:54.783646 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.783615 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp"] Jun 02 19:09:54.809119 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.809093 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb"] Jun 02 19:09:54.809466 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.809452 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d294eca9-a053-4100-a22e-67329d36c362" containerName="manager" Jun 02 19:09:54.809521 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.809468 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d294eca9-a053-4100-a22e-67329d36c362" containerName="manager" Jun 02 19:09:54.809559 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.809531 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="d294eca9-a053-4100-a22e-67329d36c362" containerName="manager" Jun 02 19:09:54.811567 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.811548 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" Jun 02 19:09:54.844755 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.844723 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb"] Jun 02 19:09:54.908171 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.908139 2574 status_manager.go:895] "Failed to get status for pod" podUID="d294eca9-a053-4100-a22e-67329d36c362" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-bcqfp\" is forbidden: User \"system:node:ip-10-0-138-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-102.ec2.internal' and this object" Jun 02 19:09:54.938691 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.938652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/528ae4cc-4828-44b4-981c-accde0a60458-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-68fcb\" (UID: \"528ae4cc-4828-44b4-981c-accde0a60458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" Jun 02 19:09:54.938869 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:54.938704 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsd7f\" (UniqueName: \"kubernetes.io/projected/528ae4cc-4828-44b4-981c-accde0a60458-kube-api-access-zsd7f\") pod \"kuadrant-operator-controller-manager-55c7f4c975-68fcb\" (UID: \"528ae4cc-4828-44b4-981c-accde0a60458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" Jun 02 19:09:55.029177 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.029094 2574 generic.go:358] "Generic (PLEG): container finished" podID="d294eca9-a053-4100-a22e-67329d36c362" containerID="2177afe4a2e6dda90f983b0a6f58166af6f4fa110c210a54c01f702754528b32" exitCode=0 Jun 02 19:09:55.029177 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.029167 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="376972d43b6ac8770f81e2b64e96677db264c0c58d124a986bb79b811a3e9421" Jun 02 19:09:55.030072 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.030004 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:55.032168 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.032144 2574 status_manager.go:895] "Failed to get status for pod" podUID="d294eca9-a053-4100-a22e-67329d36c362" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-bcqfp\" is forbidden: User \"system:node:ip-10-0-138-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-102.ec2.internal' and this object" Jun 02 19:09:55.039620 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.039598 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/528ae4cc-4828-44b4-981c-accde0a60458-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-68fcb\" (UID: \"528ae4cc-4828-44b4-981c-accde0a60458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" Jun 02 19:09:55.039716 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.039640 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zsd7f\" (UniqueName: \"kubernetes.io/projected/528ae4cc-4828-44b4-981c-accde0a60458-kube-api-access-zsd7f\") pod \"kuadrant-operator-controller-manager-55c7f4c975-68fcb\" (UID: \"528ae4cc-4828-44b4-981c-accde0a60458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" Jun 02 19:09:55.040011 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.039990 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/528ae4cc-4828-44b4-981c-accde0a60458-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-68fcb\" (UID: \"528ae4cc-4828-44b4-981c-accde0a60458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" Jun 02 19:09:55.051027 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.051002 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsd7f\" (UniqueName: \"kubernetes.io/projected/528ae4cc-4828-44b4-981c-accde0a60458-kube-api-access-zsd7f\") pod \"kuadrant-operator-controller-manager-55c7f4c975-68fcb\" (UID: \"528ae4cc-4828-44b4-981c-accde0a60458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" Jun 02 19:09:55.140167 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.140131 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d294eca9-a053-4100-a22e-67329d36c362-extensions-socket-volume\") pod \"d294eca9-a053-4100-a22e-67329d36c362\" (UID: \"d294eca9-a053-4100-a22e-67329d36c362\") " Jun 02 19:09:55.140343 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.140277 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td5hj\" (UniqueName: \"kubernetes.io/projected/d294eca9-a053-4100-a22e-67329d36c362-kube-api-access-td5hj\") pod \"d294eca9-a053-4100-a22e-67329d36c362\" (UID: \"d294eca9-a053-4100-a22e-67329d36c362\") " Jun 02 19:09:55.140732 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.140704 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d294eca9-a053-4100-a22e-67329d36c362-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d294eca9-a053-4100-a22e-67329d36c362" (UID: "d294eca9-a053-4100-a22e-67329d36c362"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 19:09:55.142765 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.142739 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d294eca9-a053-4100-a22e-67329d36c362-kube-api-access-td5hj" (OuterVolumeSpecName: "kube-api-access-td5hj") pod "d294eca9-a053-4100-a22e-67329d36c362" (UID: "d294eca9-a053-4100-a22e-67329d36c362"). InnerVolumeSpecName "kube-api-access-td5hj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:09:55.158010 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.157984 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" Jun 02 19:09:55.242011 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.241957 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d294eca9-a053-4100-a22e-67329d36c362-extensions-socket-volume\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:09:55.242011 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.241998 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td5hj\" (UniqueName: \"kubernetes.io/projected/d294eca9-a053-4100-a22e-67329d36c362-kube-api-access-td5hj\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:09:55.332668 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.332552 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb"] Jun 02 19:09:55.335331 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:09:55.335301 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod528ae4cc_4828_44b4_981c_accde0a60458.slice/crio-e6cba1b48ada40106e387fd458268d670c7741bc6b002cf78fa7916ade8e5085 WatchSource:0}: Error finding container e6cba1b48ada40106e387fd458268d670c7741bc6b002cf78fa7916ade8e5085: Status 404 returned error can't find the container with id e6cba1b48ada40106e387fd458268d670c7741bc6b002cf78fa7916ade8e5085 Jun 02 19:09:55.684494 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:55.684443 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d294eca9-a053-4100-a22e-67329d36c362" path="/var/lib/kubelet/pods/d294eca9-a053-4100-a22e-67329d36c362/volumes" Jun 02 19:09:56.037004 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:56.036918 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" event={"ID":"528ae4cc-4828-44b4-981c-accde0a60458","Type":"ContainerStarted","Data":"b01889035e7c0a010b4939da119ed98e8b2da09b67709d9a305ced33ec8bbcdb"} Jun 02 19:09:56.037004 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:56.036944 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-bcqfp" Jun 02 19:09:56.037004 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:56.036970 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" event={"ID":"528ae4cc-4828-44b4-981c-accde0a60458","Type":"ContainerStarted","Data":"e6cba1b48ada40106e387fd458268d670c7741bc6b002cf78fa7916ade8e5085"} Jun 02 19:09:56.037558 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:56.037023 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" Jun 02 19:09:56.064649 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:09:56.064605 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" podStartSLOduration=2.064587485 podStartE2EDuration="2.064587485s" podCreationTimestamp="2026-06-02 19:09:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:09:56.062686116 +0000 UTC m=+358.880655841" watchObservedRunningTime="2026-06-02 19:09:56.064587485 +0000 UTC m=+358.882557209" Jun 02 19:10:07.043282 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:07.043249 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-68fcb" Jun 02 19:10:12.099875 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:12.099844 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" event={"ID":"d13a3fb7-eb6a-4e14-b984-612e04083bde","Type":"ContainerStarted","Data":"e386648136952159b4032f7a5daa7e98e6fc16e0a0a7c018ffe2e0a77533bce1"} Jun 02 19:10:12.116644 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:12.116601 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-jrd8w" podStartSLOduration=1.80915386 podStartE2EDuration="28.116588055s" podCreationTimestamp="2026-06-02 19:09:44 +0000 UTC" firstStartedPulling="2026-06-02 19:09:45.043199105 +0000 UTC m=+347.861168808" lastFinishedPulling="2026-06-02 19:10:11.350633301 +0000 UTC m=+374.168603003" observedRunningTime="2026-06-02 19:10:12.114394377 +0000 UTC m=+374.932364100" watchObservedRunningTime="2026-06-02 19:10:12.116588055 +0000 UTC m=+374.934557778" Jun 02 19:10:27.934256 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:27.934205 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-p5qkc"] Jun 02 19:10:27.996677 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:27.996640 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-p5qkc"] Jun 02 19:10:27.996835 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:27.996750 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" Jun 02 19:10:27.998819 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:27.998798 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hncxm\"" Jun 02 19:10:28.063363 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.063333 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-2zd56"] Jun 02 19:10:28.066677 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.066660 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-2zd56" Jun 02 19:10:28.071672 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.071649 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-2zd56"] Jun 02 19:10:28.117284 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.117256 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmhhk\" (UniqueName: \"kubernetes.io/projected/6d3a1aea-df8f-404f-8072-38f956dc4806-kube-api-access-jmhhk\") pod \"authorino-7498df8756-2zd56\" (UID: \"6d3a1aea-df8f-404f-8072-38f956dc4806\") " pod="kuadrant-system/authorino-7498df8756-2zd56" Jun 02 19:10:28.117446 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.117298 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crhjd\" (UniqueName: \"kubernetes.io/projected/ea003f5b-63cd-4a6e-8ea2-1f8da006bd27-kube-api-access-crhjd\") pod \"authorino-f99f4b5cd-p5qkc\" (UID: \"ea003f5b-63cd-4a6e-8ea2-1f8da006bd27\") " pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" Jun 02 19:10:28.217860 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.217778 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jmhhk\" (UniqueName: \"kubernetes.io/projected/6d3a1aea-df8f-404f-8072-38f956dc4806-kube-api-access-jmhhk\") pod \"authorino-7498df8756-2zd56\" (UID: \"6d3a1aea-df8f-404f-8072-38f956dc4806\") " pod="kuadrant-system/authorino-7498df8756-2zd56" Jun 02 19:10:28.217860 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.217821 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-crhjd\" (UniqueName: \"kubernetes.io/projected/ea003f5b-63cd-4a6e-8ea2-1f8da006bd27-kube-api-access-crhjd\") pod \"authorino-f99f4b5cd-p5qkc\" (UID: \"ea003f5b-63cd-4a6e-8ea2-1f8da006bd27\") " pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" Jun 02 19:10:28.225368 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.225342 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmhhk\" (UniqueName: \"kubernetes.io/projected/6d3a1aea-df8f-404f-8072-38f956dc4806-kube-api-access-jmhhk\") pod \"authorino-7498df8756-2zd56\" (UID: \"6d3a1aea-df8f-404f-8072-38f956dc4806\") " pod="kuadrant-system/authorino-7498df8756-2zd56" Jun 02 19:10:28.225612 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.225592 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crhjd\" (UniqueName: \"kubernetes.io/projected/ea003f5b-63cd-4a6e-8ea2-1f8da006bd27-kube-api-access-crhjd\") pod \"authorino-f99f4b5cd-p5qkc\" (UID: \"ea003f5b-63cd-4a6e-8ea2-1f8da006bd27\") " pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" Jun 02 19:10:28.306459 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.306416 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" Jun 02 19:10:28.376518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.376491 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-2zd56" Jun 02 19:10:28.430945 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.430910 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-p5qkc"] Jun 02 19:10:28.434896 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:10:28.434863 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea003f5b_63cd_4a6e_8ea2_1f8da006bd27.slice/crio-f7228a166c452abf121643d46685f03582d82e3f2ede4be94ef8ef92908cfd73 WatchSource:0}: Error finding container f7228a166c452abf121643d46685f03582d82e3f2ede4be94ef8ef92908cfd73: Status 404 returned error can't find the container with id f7228a166c452abf121643d46685f03582d82e3f2ede4be94ef8ef92908cfd73 Jun 02 19:10:28.503544 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:28.503516 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-2zd56"] Jun 02 19:10:28.506858 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:10:28.506827 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d3a1aea_df8f_404f_8072_38f956dc4806.slice/crio-6343f3bc1e65657e0e72b35e8fdbcd430430b20c62a00bb46d87da8414b58959 WatchSource:0}: Error finding container 6343f3bc1e65657e0e72b35e8fdbcd430430b20c62a00bb46d87da8414b58959: Status 404 returned error can't find the container with id 6343f3bc1e65657e0e72b35e8fdbcd430430b20c62a00bb46d87da8414b58959 Jun 02 19:10:29.163598 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:29.163415 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" event={"ID":"ea003f5b-63cd-4a6e-8ea2-1f8da006bd27","Type":"ContainerStarted","Data":"f7228a166c452abf121643d46685f03582d82e3f2ede4be94ef8ef92908cfd73"} Jun 02 19:10:29.165099 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:29.165056 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-2zd56" event={"ID":"6d3a1aea-df8f-404f-8072-38f956dc4806","Type":"ContainerStarted","Data":"6343f3bc1e65657e0e72b35e8fdbcd430430b20c62a00bb46d87da8414b58959"} Jun 02 19:10:32.179785 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:32.179692 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-2zd56" event={"ID":"6d3a1aea-df8f-404f-8072-38f956dc4806","Type":"ContainerStarted","Data":"65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231"} Jun 02 19:10:32.181006 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:32.180983 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" event={"ID":"ea003f5b-63cd-4a6e-8ea2-1f8da006bd27","Type":"ContainerStarted","Data":"527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3"} Jun 02 19:10:32.211890 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:32.211840 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-2zd56" podStartSLOduration=0.820539197 podStartE2EDuration="4.211827828s" podCreationTimestamp="2026-06-02 19:10:28 +0000 UTC" firstStartedPulling="2026-06-02 19:10:28.50820268 +0000 UTC m=+391.326172383" lastFinishedPulling="2026-06-02 19:10:31.899491312 +0000 UTC m=+394.717461014" observedRunningTime="2026-06-02 19:10:32.209604585 +0000 UTC m=+395.027574309" watchObservedRunningTime="2026-06-02 19:10:32.211827828 +0000 UTC m=+395.029797551" Jun 02 19:10:32.242493 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:32.242449 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" podStartSLOduration=1.7683565940000001 podStartE2EDuration="5.242435302s" podCreationTimestamp="2026-06-02 19:10:27 +0000 UTC" firstStartedPulling="2026-06-02 19:10:28.436471819 +0000 UTC m=+391.254441539" lastFinishedPulling="2026-06-02 19:10:31.910550532 +0000 UTC m=+394.728520247" observedRunningTime="2026-06-02 19:10:32.241098581 +0000 UTC m=+395.059068304" watchObservedRunningTime="2026-06-02 19:10:32.242435302 +0000 UTC m=+395.060405078" Jun 02 19:10:32.255962 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:32.255934 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-p5qkc"] Jun 02 19:10:34.188338 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:34.188299 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" podUID="ea003f5b-63cd-4a6e-8ea2-1f8da006bd27" containerName="authorino" containerID="cri-o://527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3" gracePeriod=30 Jun 02 19:10:34.443337 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:34.443270 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" Jun 02 19:10:34.582340 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:34.582306 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crhjd\" (UniqueName: \"kubernetes.io/projected/ea003f5b-63cd-4a6e-8ea2-1f8da006bd27-kube-api-access-crhjd\") pod \"ea003f5b-63cd-4a6e-8ea2-1f8da006bd27\" (UID: \"ea003f5b-63cd-4a6e-8ea2-1f8da006bd27\") " Jun 02 19:10:34.584387 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:34.584354 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea003f5b-63cd-4a6e-8ea2-1f8da006bd27-kube-api-access-crhjd" (OuterVolumeSpecName: "kube-api-access-crhjd") pod "ea003f5b-63cd-4a6e-8ea2-1f8da006bd27" (UID: "ea003f5b-63cd-4a6e-8ea2-1f8da006bd27"). InnerVolumeSpecName "kube-api-access-crhjd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:10:34.682926 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:34.682899 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crhjd\" (UniqueName: \"kubernetes.io/projected/ea003f5b-63cd-4a6e-8ea2-1f8da006bd27-kube-api-access-crhjd\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:10:35.194110 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:35.194073 2574 generic.go:358] "Generic (PLEG): container finished" podID="ea003f5b-63cd-4a6e-8ea2-1f8da006bd27" containerID="527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3" exitCode=0 Jun 02 19:10:35.194614 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:35.194126 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" Jun 02 19:10:35.194614 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:35.194142 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" event={"ID":"ea003f5b-63cd-4a6e-8ea2-1f8da006bd27","Type":"ContainerDied","Data":"527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3"} Jun 02 19:10:35.194614 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:35.194170 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-p5qkc" event={"ID":"ea003f5b-63cd-4a6e-8ea2-1f8da006bd27","Type":"ContainerDied","Data":"f7228a166c452abf121643d46685f03582d82e3f2ede4be94ef8ef92908cfd73"} Jun 02 19:10:35.194614 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:35.194184 2574 scope.go:117] "RemoveContainer" containerID="527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3" Jun 02 19:10:35.202885 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:35.202868 2574 scope.go:117] "RemoveContainer" containerID="527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3" Jun 02 19:10:35.203139 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:10:35.203123 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3\": container with ID starting with 527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3 not found: ID does not exist" containerID="527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3" Jun 02 19:10:35.203186 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:35.203146 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3"} err="failed to get container status \"527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3\": rpc error: code = NotFound desc = could not find container \"527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3\": container with ID starting with 527de73661422f2d07b4ae0119bf5787ed5ebffe9df70414644923b56a5308e3 not found: ID does not exist" Jun 02 19:10:35.216116 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:35.216090 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-p5qkc"] Jun 02 19:10:35.217628 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:35.217609 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-p5qkc"] Jun 02 19:10:35.683274 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:10:35.683245 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea003f5b-63cd-4a6e-8ea2-1f8da006bd27" path="/var/lib/kubelet/pods/ea003f5b-63cd-4a6e-8ea2-1f8da006bd27/volumes" Jun 02 19:11:23.774125 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.774043 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:11:23.774588 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.774395 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea003f5b-63cd-4a6e-8ea2-1f8da006bd27" containerName="authorino" Jun 02 19:11:23.774588 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.774407 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea003f5b-63cd-4a6e-8ea2-1f8da006bd27" containerName="authorino" Jun 02 19:11:23.774588 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.774484 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="ea003f5b-63cd-4a6e-8ea2-1f8da006bd27" containerName="authorino" Jun 02 19:11:23.777635 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.777613 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:11:23.779933 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.779909 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 19:11:23.780039 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.779911 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 19:11:23.780039 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.779953 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 19:11:23.780556 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.780540 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-v9wdh\"" Jun 02 19:11:23.786276 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.786254 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:11:23.890428 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.890401 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc78t\" (UniqueName: \"kubernetes.io/projected/7484c17d-2c0e-4684-89e1-8695190dde0a-kube-api-access-wc78t\") pod \"maas-keycloak-0\" (UID: \"7484c17d-2c0e-4684-89e1-8695190dde0a\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:11:23.990819 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.990787 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wc78t\" (UniqueName: \"kubernetes.io/projected/7484c17d-2c0e-4684-89e1-8695190dde0a-kube-api-access-wc78t\") pod \"maas-keycloak-0\" (UID: \"7484c17d-2c0e-4684-89e1-8695190dde0a\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:11:23.998955 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:23.998922 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc78t\" (UniqueName: \"kubernetes.io/projected/7484c17d-2c0e-4684-89e1-8695190dde0a-kube-api-access-wc78t\") pod \"maas-keycloak-0\" (UID: \"7484c17d-2c0e-4684-89e1-8695190dde0a\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:11:24.087831 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:24.087747 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:11:24.214292 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:24.214268 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:11:24.217051 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:11:24.217025 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7484c17d_2c0e_4684_89e1_8695190dde0a.slice/crio-dcd74361fb1c3ab6a9f4c1ce6be6c3426802416901f2ccbfc427299a718edce6 WatchSource:0}: Error finding container dcd74361fb1c3ab6a9f4c1ce6be6c3426802416901f2ccbfc427299a718edce6: Status 404 returned error can't find the container with id dcd74361fb1c3ab6a9f4c1ce6be6c3426802416901f2ccbfc427299a718edce6 Jun 02 19:11:24.370285 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:24.370174 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7484c17d-2c0e-4684-89e1-8695190dde0a","Type":"ContainerStarted","Data":"dcd74361fb1c3ab6a9f4c1ce6be6c3426802416901f2ccbfc427299a718edce6"} Jun 02 19:11:30.393073 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:30.393045 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7484c17d-2c0e-4684-89e1-8695190dde0a","Type":"ContainerStarted","Data":"cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07"} Jun 02 19:11:30.409199 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:30.409149 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.297019726 podStartE2EDuration="7.409131554s" podCreationTimestamp="2026-06-02 19:11:23 +0000 UTC" firstStartedPulling="2026-06-02 19:11:24.218832038 +0000 UTC m=+447.036801743" lastFinishedPulling="2026-06-02 19:11:30.330943869 +0000 UTC m=+453.148913571" observedRunningTime="2026-06-02 19:11:30.408191241 +0000 UTC m=+453.226161002" watchObservedRunningTime="2026-06-02 19:11:30.409131554 +0000 UTC m=+453.227101279" Jun 02 19:11:31.088509 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:31.088466 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 19:11:31.090192 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:31.090153 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:32.088297 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:32.088238 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:33.089116 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:33.089071 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:34.088830 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:34.088785 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 19:11:34.089102 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:34.089064 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:35.088496 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:35.088439 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:36.088417 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:36.088376 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:37.088866 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:37.088818 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:38.088837 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:38.088789 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:39.088180 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:39.088138 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:40.088478 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:40.088420 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:41.088129 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:41.088071 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:42.088087 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:42.088041 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 02 19:11:43.199154 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:43.199116 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 19:11:43.213181 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:43.213130 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 19:11:53.205874 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:53.205826 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 19:11:54.069543 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.069511 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-nf577"] Jun 02 19:11:54.079495 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.079471 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-nf577" Jun 02 19:11:54.080029 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.080008 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-nf577"] Jun 02 19:11:54.168811 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.168783 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whwzt\" (UniqueName: \"kubernetes.io/projected/4ee01478-2bfb-4403-bf11-f8366facd14a-kube-api-access-whwzt\") pod \"authorino-8b475cf9f-nf577\" (UID: \"4ee01478-2bfb-4403-bf11-f8366facd14a\") " pod="kuadrant-system/authorino-8b475cf9f-nf577" Jun 02 19:11:54.269687 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.269661 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-whwzt\" (UniqueName: \"kubernetes.io/projected/4ee01478-2bfb-4403-bf11-f8366facd14a-kube-api-access-whwzt\") pod \"authorino-8b475cf9f-nf577\" (UID: \"4ee01478-2bfb-4403-bf11-f8366facd14a\") " pod="kuadrant-system/authorino-8b475cf9f-nf577" Jun 02 19:11:54.277593 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.277575 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whwzt\" (UniqueName: \"kubernetes.io/projected/4ee01478-2bfb-4403-bf11-f8366facd14a-kube-api-access-whwzt\") pod \"authorino-8b475cf9f-nf577\" (UID: \"4ee01478-2bfb-4403-bf11-f8366facd14a\") " pod="kuadrant-system/authorino-8b475cf9f-nf577" Jun 02 19:11:54.299718 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.299688 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-nf577"] Jun 02 19:11:54.299859 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.299848 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-nf577" Jun 02 19:11:54.328898 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.328813 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79b95c4569-498c9"] Jun 02 19:11:54.333169 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.333153 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79b95c4569-498c9" Jun 02 19:11:54.342672 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.342652 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79b95c4569-498c9"] Jun 02 19:11:54.370579 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.370555 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jnzw\" (UniqueName: \"kubernetes.io/projected/932ac148-f732-4eed-af5e-d92ecf548e79-kube-api-access-2jnzw\") pod \"authorino-79b95c4569-498c9\" (UID: \"932ac148-f732-4eed-af5e-d92ecf548e79\") " pod="kuadrant-system/authorino-79b95c4569-498c9" Jun 02 19:11:54.429540 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.429512 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-nf577"] Jun 02 19:11:54.432510 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:11:54.432481 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ee01478_2bfb_4403_bf11_f8366facd14a.slice/crio-5e602e15084f89b7b5a96aa7addbedcbde8d726e50c93b6bfbdaca554ad38020 WatchSource:0}: Error finding container 5e602e15084f89b7b5a96aa7addbedcbde8d726e50c93b6bfbdaca554ad38020: Status 404 returned error can't find the container with id 5e602e15084f89b7b5a96aa7addbedcbde8d726e50c93b6bfbdaca554ad38020 Jun 02 19:11:54.471448 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.471423 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2jnzw\" (UniqueName: \"kubernetes.io/projected/932ac148-f732-4eed-af5e-d92ecf548e79-kube-api-access-2jnzw\") pod \"authorino-79b95c4569-498c9\" (UID: \"932ac148-f732-4eed-af5e-d92ecf548e79\") " pod="kuadrant-system/authorino-79b95c4569-498c9" Jun 02 19:11:54.481890 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.481867 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jnzw\" (UniqueName: \"kubernetes.io/projected/932ac148-f732-4eed-af5e-d92ecf548e79-kube-api-access-2jnzw\") pod \"authorino-79b95c4569-498c9\" (UID: \"932ac148-f732-4eed-af5e-d92ecf548e79\") " pod="kuadrant-system/authorino-79b95c4569-498c9" Jun 02 19:11:54.497279 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.497250 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-nf577" event={"ID":"4ee01478-2bfb-4403-bf11-f8366facd14a","Type":"ContainerStarted","Data":"5e602e15084f89b7b5a96aa7addbedcbde8d726e50c93b6bfbdaca554ad38020"} Jun 02 19:11:54.508267 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.508232 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79b95c4569-498c9"] Jun 02 19:11:54.508458 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.508447 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79b95c4569-498c9" Jun 02 19:11:54.534627 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.534603 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c69447669-8lr7j"] Jun 02 19:11:54.539245 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.539211 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c69447669-8lr7j" Jun 02 19:11:54.541532 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.541500 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 02 19:11:54.544549 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.544513 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c69447669-8lr7j"] Jun 02 19:11:54.572310 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.572208 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7grd\" (UniqueName: \"kubernetes.io/projected/443df020-d00d-47df-8604-9c3d5bdfaeb1-kube-api-access-c7grd\") pod \"authorino-5c69447669-8lr7j\" (UID: \"443df020-d00d-47df-8604-9c3d5bdfaeb1\") " pod="kuadrant-system/authorino-5c69447669-8lr7j" Jun 02 19:11:54.572444 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.572339 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/443df020-d00d-47df-8604-9c3d5bdfaeb1-tls-cert\") pod \"authorino-5c69447669-8lr7j\" (UID: \"443df020-d00d-47df-8604-9c3d5bdfaeb1\") " pod="kuadrant-system/authorino-5c69447669-8lr7j" Jun 02 19:11:54.634519 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.634496 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79b95c4569-498c9"] Jun 02 19:11:54.636511 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:11:54.636486 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod932ac148_f732_4eed_af5e_d92ecf548e79.slice/crio-1a6d423dbf5e53ebcfe847f7aca9aa7e7ae2062b1d1f03c9de2c11b93ccffd92 WatchSource:0}: Error finding container 1a6d423dbf5e53ebcfe847f7aca9aa7e7ae2062b1d1f03c9de2c11b93ccffd92: Status 404 returned error can't find the container with id 1a6d423dbf5e53ebcfe847f7aca9aa7e7ae2062b1d1f03c9de2c11b93ccffd92 Jun 02 19:11:54.673697 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.673673 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c7grd\" (UniqueName: \"kubernetes.io/projected/443df020-d00d-47df-8604-9c3d5bdfaeb1-kube-api-access-c7grd\") pod \"authorino-5c69447669-8lr7j\" (UID: \"443df020-d00d-47df-8604-9c3d5bdfaeb1\") " pod="kuadrant-system/authorino-5c69447669-8lr7j" Jun 02 19:11:54.673796 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.673710 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/443df020-d00d-47df-8604-9c3d5bdfaeb1-tls-cert\") pod \"authorino-5c69447669-8lr7j\" (UID: \"443df020-d00d-47df-8604-9c3d5bdfaeb1\") " pod="kuadrant-system/authorino-5c69447669-8lr7j" Jun 02 19:11:54.675870 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.675852 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/443df020-d00d-47df-8604-9c3d5bdfaeb1-tls-cert\") pod \"authorino-5c69447669-8lr7j\" (UID: \"443df020-d00d-47df-8604-9c3d5bdfaeb1\") " pod="kuadrant-system/authorino-5c69447669-8lr7j" Jun 02 19:11:54.680639 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.680619 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7grd\" (UniqueName: \"kubernetes.io/projected/443df020-d00d-47df-8604-9c3d5bdfaeb1-kube-api-access-c7grd\") pod \"authorino-5c69447669-8lr7j\" (UID: \"443df020-d00d-47df-8604-9c3d5bdfaeb1\") " pod="kuadrant-system/authorino-5c69447669-8lr7j" Jun 02 19:11:54.850265 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.850196 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c69447669-8lr7j" Jun 02 19:11:54.975963 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:54.975932 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c69447669-8lr7j"] Jun 02 19:11:54.979134 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:11:54.979113 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod443df020_d00d_47df_8604_9c3d5bdfaeb1.slice/crio-b205546d207546795ca12c86f0fb52be1f28505e1f89c6ec4677816139388a2f WatchSource:0}: Error finding container b205546d207546795ca12c86f0fb52be1f28505e1f89c6ec4677816139388a2f: Status 404 returned error can't find the container with id b205546d207546795ca12c86f0fb52be1f28505e1f89c6ec4677816139388a2f Jun 02 19:11:55.502173 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.502134 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79b95c4569-498c9" event={"ID":"932ac148-f732-4eed-af5e-d92ecf548e79","Type":"ContainerStarted","Data":"a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8"} Jun 02 19:11:55.502173 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.502173 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79b95c4569-498c9" event={"ID":"932ac148-f732-4eed-af5e-d92ecf548e79","Type":"ContainerStarted","Data":"1a6d423dbf5e53ebcfe847f7aca9aa7e7ae2062b1d1f03c9de2c11b93ccffd92"} Jun 02 19:11:55.502647 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.502194 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-79b95c4569-498c9" podUID="932ac148-f732-4eed-af5e-d92ecf548e79" containerName="authorino" containerID="cri-o://a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8" gracePeriod=30 Jun 02 19:11:55.503549 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.503528 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-nf577" event={"ID":"4ee01478-2bfb-4403-bf11-f8366facd14a","Type":"ContainerStarted","Data":"bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb"} Jun 02 19:11:55.503679 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.503591 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-nf577" podUID="4ee01478-2bfb-4403-bf11-f8366facd14a" containerName="authorino" containerID="cri-o://bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb" gracePeriod=30 Jun 02 19:11:55.504962 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.504939 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c69447669-8lr7j" event={"ID":"443df020-d00d-47df-8604-9c3d5bdfaeb1","Type":"ContainerStarted","Data":"388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02"} Jun 02 19:11:55.505079 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.504968 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c69447669-8lr7j" event={"ID":"443df020-d00d-47df-8604-9c3d5bdfaeb1","Type":"ContainerStarted","Data":"b205546d207546795ca12c86f0fb52be1f28505e1f89c6ec4677816139388a2f"} Jun 02 19:11:55.518574 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.518534 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-79b95c4569-498c9" podStartSLOduration=1.183138235 podStartE2EDuration="1.518520596s" podCreationTimestamp="2026-06-02 19:11:54 +0000 UTC" firstStartedPulling="2026-06-02 19:11:54.637890809 +0000 UTC m=+477.455860512" lastFinishedPulling="2026-06-02 19:11:54.973273156 +0000 UTC m=+477.791242873" observedRunningTime="2026-06-02 19:11:55.517402165 +0000 UTC m=+478.335371886" watchObservedRunningTime="2026-06-02 19:11:55.518520596 +0000 UTC m=+478.336490321" Jun 02 19:11:55.532968 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.532933 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-5c69447669-8lr7j" podStartSLOduration=1.211999782 podStartE2EDuration="1.532920678s" podCreationTimestamp="2026-06-02 19:11:54 +0000 UTC" firstStartedPulling="2026-06-02 19:11:54.980402731 +0000 UTC m=+477.798372434" lastFinishedPulling="2026-06-02 19:11:55.301323626 +0000 UTC m=+478.119293330" observedRunningTime="2026-06-02 19:11:55.532264254 +0000 UTC m=+478.350233977" watchObservedRunningTime="2026-06-02 19:11:55.532920678 +0000 UTC m=+478.350890402" Jun 02 19:11:55.547661 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.547613 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-nf577" podStartSLOduration=1.073710028 podStartE2EDuration="1.547601559s" podCreationTimestamp="2026-06-02 19:11:54 +0000 UTC" firstStartedPulling="2026-06-02 19:11:54.434048741 +0000 UTC m=+477.252018442" lastFinishedPulling="2026-06-02 19:11:54.90794027 +0000 UTC m=+477.725909973" observedRunningTime="2026-06-02 19:11:55.545911091 +0000 UTC m=+478.363880815" watchObservedRunningTime="2026-06-02 19:11:55.547601559 +0000 UTC m=+478.365571284" Jun 02 19:11:55.561770 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.561699 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-2zd56"] Jun 02 19:11:55.561949 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.561925 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-2zd56" podUID="6d3a1aea-df8f-404f-8072-38f956dc4806" containerName="authorino" containerID="cri-o://65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231" gracePeriod=30 Jun 02 19:11:55.824576 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.824551 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-nf577" Jun 02 19:11:55.839415 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.839394 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-2zd56" Jun 02 19:11:55.853118 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.853097 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79b95c4569-498c9" Jun 02 19:11:55.883773 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.883751 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whwzt\" (UniqueName: \"kubernetes.io/projected/4ee01478-2bfb-4403-bf11-f8366facd14a-kube-api-access-whwzt\") pod \"4ee01478-2bfb-4403-bf11-f8366facd14a\" (UID: \"4ee01478-2bfb-4403-bf11-f8366facd14a\") " Jun 02 19:11:55.883879 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.883782 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jnzw\" (UniqueName: \"kubernetes.io/projected/932ac148-f732-4eed-af5e-d92ecf548e79-kube-api-access-2jnzw\") pod \"932ac148-f732-4eed-af5e-d92ecf548e79\" (UID: \"932ac148-f732-4eed-af5e-d92ecf548e79\") " Jun 02 19:11:55.883879 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.883817 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmhhk\" (UniqueName: \"kubernetes.io/projected/6d3a1aea-df8f-404f-8072-38f956dc4806-kube-api-access-jmhhk\") pod \"6d3a1aea-df8f-404f-8072-38f956dc4806\" (UID: \"6d3a1aea-df8f-404f-8072-38f956dc4806\") " Jun 02 19:11:55.885766 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.885732 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ee01478-2bfb-4403-bf11-f8366facd14a-kube-api-access-whwzt" (OuterVolumeSpecName: "kube-api-access-whwzt") pod "4ee01478-2bfb-4403-bf11-f8366facd14a" (UID: "4ee01478-2bfb-4403-bf11-f8366facd14a"). InnerVolumeSpecName "kube-api-access-whwzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:11:55.885873 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.885777 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d3a1aea-df8f-404f-8072-38f956dc4806-kube-api-access-jmhhk" (OuterVolumeSpecName: "kube-api-access-jmhhk") pod "6d3a1aea-df8f-404f-8072-38f956dc4806" (UID: "6d3a1aea-df8f-404f-8072-38f956dc4806"). InnerVolumeSpecName "kube-api-access-jmhhk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:11:55.885873 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.885838 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/932ac148-f732-4eed-af5e-d92ecf548e79-kube-api-access-2jnzw" (OuterVolumeSpecName: "kube-api-access-2jnzw") pod "932ac148-f732-4eed-af5e-d92ecf548e79" (UID: "932ac148-f732-4eed-af5e-d92ecf548e79"). InnerVolumeSpecName "kube-api-access-2jnzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:11:55.985075 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.985049 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whwzt\" (UniqueName: \"kubernetes.io/projected/4ee01478-2bfb-4403-bf11-f8366facd14a-kube-api-access-whwzt\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:11:55.985075 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.985072 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jnzw\" (UniqueName: \"kubernetes.io/projected/932ac148-f732-4eed-af5e-d92ecf548e79-kube-api-access-2jnzw\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:11:55.985213 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:55.985082 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmhhk\" (UniqueName: \"kubernetes.io/projected/6d3a1aea-df8f-404f-8072-38f956dc4806-kube-api-access-jmhhk\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:11:56.515581 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.515546 2574 generic.go:358] "Generic (PLEG): container finished" podID="6d3a1aea-df8f-404f-8072-38f956dc4806" containerID="65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231" exitCode=0 Jun 02 19:11:56.516099 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.515611 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-2zd56" Jun 02 19:11:56.516099 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.515630 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-2zd56" event={"ID":"6d3a1aea-df8f-404f-8072-38f956dc4806","Type":"ContainerDied","Data":"65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231"} Jun 02 19:11:56.516099 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.515664 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-2zd56" event={"ID":"6d3a1aea-df8f-404f-8072-38f956dc4806","Type":"ContainerDied","Data":"6343f3bc1e65657e0e72b35e8fdbcd430430b20c62a00bb46d87da8414b58959"} Jun 02 19:11:56.516099 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.515684 2574 scope.go:117] "RemoveContainer" containerID="65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231" Jun 02 19:11:56.516950 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.516766 2574 generic.go:358] "Generic (PLEG): container finished" podID="932ac148-f732-4eed-af5e-d92ecf548e79" containerID="a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8" exitCode=0 Jun 02 19:11:56.516950 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.516835 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79b95c4569-498c9" Jun 02 19:11:56.516950 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.516854 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79b95c4569-498c9" event={"ID":"932ac148-f732-4eed-af5e-d92ecf548e79","Type":"ContainerDied","Data":"a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8"} Jun 02 19:11:56.516950 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.516890 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79b95c4569-498c9" event={"ID":"932ac148-f732-4eed-af5e-d92ecf548e79","Type":"ContainerDied","Data":"1a6d423dbf5e53ebcfe847f7aca9aa7e7ae2062b1d1f03c9de2c11b93ccffd92"} Jun 02 19:11:56.518012 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.517991 2574 generic.go:358] "Generic (PLEG): container finished" podID="4ee01478-2bfb-4403-bf11-f8366facd14a" containerID="bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb" exitCode=0 Jun 02 19:11:56.518128 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.518054 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-nf577" Jun 02 19:11:56.518128 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.518077 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-nf577" event={"ID":"4ee01478-2bfb-4403-bf11-f8366facd14a","Type":"ContainerDied","Data":"bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb"} Jun 02 19:11:56.518128 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.518106 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-nf577" event={"ID":"4ee01478-2bfb-4403-bf11-f8366facd14a","Type":"ContainerDied","Data":"5e602e15084f89b7b5a96aa7addbedcbde8d726e50c93b6bfbdaca554ad38020"} Jun 02 19:11:56.525597 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.525572 2574 scope.go:117] "RemoveContainer" containerID="65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231" Jun 02 19:11:56.525923 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:11:56.525901 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231\": container with ID starting with 65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231 not found: ID does not exist" containerID="65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231" Jun 02 19:11:56.526020 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.525930 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231"} err="failed to get container status \"65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231\": rpc error: code = NotFound desc = could not find container \"65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231\": container with ID starting with 65696b3eecedcae30be67544c9d30ed7dd32584419eaf726df486c17ade62231 not found: ID does not exist" Jun 02 19:11:56.526020 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.525947 2574 scope.go:117] "RemoveContainer" containerID="a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8" Jun 02 19:11:56.533891 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.533881 2574 scope.go:117] "RemoveContainer" containerID="a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8" Jun 02 19:11:56.534185 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:11:56.534164 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8\": container with ID starting with a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8 not found: ID does not exist" containerID="a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8" Jun 02 19:11:56.534298 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.534188 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8"} err="failed to get container status \"a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8\": rpc error: code = NotFound desc = could not find container \"a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8\": container with ID starting with a4c7181476cf19df3fd8267e8a222c9ada32bf6b42e923feb40389518259daf8 not found: ID does not exist" Jun 02 19:11:56.534298 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.534209 2574 scope.go:117] "RemoveContainer" containerID="bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb" Jun 02 19:11:56.541113 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.541095 2574 scope.go:117] "RemoveContainer" containerID="bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb" Jun 02 19:11:56.541352 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:11:56.541334 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb\": container with ID starting with bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb not found: ID does not exist" containerID="bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb" Jun 02 19:11:56.541420 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.541356 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb"} err="failed to get container status \"bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb\": rpc error: code = NotFound desc = could not find container \"bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb\": container with ID starting with bee57879fc75fc25ee7c4afb9f44300bbe3cb04a49cf5476a20b68e0e1a7a1cb not found: ID does not exist" Jun 02 19:11:56.549768 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.549740 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-2zd56"] Jun 02 19:11:56.553502 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.553484 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-2zd56"] Jun 02 19:11:56.568721 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.568700 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79b95c4569-498c9"] Jun 02 19:11:56.572705 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.572686 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79b95c4569-498c9"] Jun 02 19:11:56.586820 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.586794 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-nf577"] Jun 02 19:11:56.592861 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:56.592843 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-nf577"] Jun 02 19:11:57.683131 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:57.683109 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ee01478-2bfb-4403-bf11-f8366facd14a" path="/var/lib/kubelet/pods/4ee01478-2bfb-4403-bf11-f8366facd14a/volumes" Jun 02 19:11:57.683445 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:57.683431 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d3a1aea-df8f-404f-8072-38f956dc4806" path="/var/lib/kubelet/pods/6d3a1aea-df8f-404f-8072-38f956dc4806/volumes" Jun 02 19:11:57.683709 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:11:57.683698 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="932ac148-f732-4eed-af5e-d92ecf548e79" path="/var/lib/kubelet/pods/932ac148-f732-4eed-af5e-d92ecf548e79/volumes" Jun 02 19:12:32.982273 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.982231 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-779459d849-64v4d"] Jun 02 19:12:32.982870 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.982760 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="932ac148-f732-4eed-af5e-d92ecf548e79" containerName="authorino" Jun 02 19:12:32.982870 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.982780 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="932ac148-f732-4eed-af5e-d92ecf548e79" containerName="authorino" Jun 02 19:12:32.982870 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.982817 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d3a1aea-df8f-404f-8072-38f956dc4806" containerName="authorino" Jun 02 19:12:32.982870 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.982827 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d3a1aea-df8f-404f-8072-38f956dc4806" containerName="authorino" Jun 02 19:12:32.982870 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.982847 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ee01478-2bfb-4403-bf11-f8366facd14a" containerName="authorino" Jun 02 19:12:32.982870 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.982855 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ee01478-2bfb-4403-bf11-f8366facd14a" containerName="authorino" Jun 02 19:12:32.983169 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.982937 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="6d3a1aea-df8f-404f-8072-38f956dc4806" containerName="authorino" Jun 02 19:12:32.983169 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.982953 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="932ac148-f732-4eed-af5e-d92ecf548e79" containerName="authorino" Jun 02 19:12:32.983169 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.982963 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="4ee01478-2bfb-4403-bf11-f8366facd14a" containerName="authorino" Jun 02 19:12:32.986317 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.986296 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-779459d849-64v4d" Jun 02 19:12:32.988353 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.988322 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 02 19:12:32.996924 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:32.996896 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-779459d849-64v4d"] Jun 02 19:12:33.064447 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:33.064426 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhk2f\" (UniqueName: \"kubernetes.io/projected/ca33f358-fe61-4300-8b66-6aad28787b85-kube-api-access-zhk2f\") pod \"maas-api-779459d849-64v4d\" (UID: \"ca33f358-fe61-4300-8b66-6aad28787b85\") " pod="opendatahub/maas-api-779459d849-64v4d" Jun 02 19:12:33.064556 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:33.064457 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca33f358-fe61-4300-8b66-6aad28787b85-maas-api-tls\") pod \"maas-api-779459d849-64v4d\" (UID: \"ca33f358-fe61-4300-8b66-6aad28787b85\") " pod="opendatahub/maas-api-779459d849-64v4d" Jun 02 19:12:33.165529 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:33.165507 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca33f358-fe61-4300-8b66-6aad28787b85-maas-api-tls\") pod \"maas-api-779459d849-64v4d\" (UID: \"ca33f358-fe61-4300-8b66-6aad28787b85\") " pod="opendatahub/maas-api-779459d849-64v4d" Jun 02 19:12:33.165615 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:33.165577 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zhk2f\" (UniqueName: \"kubernetes.io/projected/ca33f358-fe61-4300-8b66-6aad28787b85-kube-api-access-zhk2f\") pod \"maas-api-779459d849-64v4d\" (UID: \"ca33f358-fe61-4300-8b66-6aad28787b85\") " pod="opendatahub/maas-api-779459d849-64v4d" Jun 02 19:12:33.165661 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:12:33.165638 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 02 19:12:33.165721 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:12:33.165709 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca33f358-fe61-4300-8b66-6aad28787b85-maas-api-tls podName:ca33f358-fe61-4300-8b66-6aad28787b85 nodeName:}" failed. No retries permitted until 2026-06-02 19:12:33.665693602 +0000 UTC m=+516.483663307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca33f358-fe61-4300-8b66-6aad28787b85-maas-api-tls") pod "maas-api-779459d849-64v4d" (UID: "ca33f358-fe61-4300-8b66-6aad28787b85") : secret "maas-api-serving-cert" not found Jun 02 19:12:33.174601 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:33.174581 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhk2f\" (UniqueName: \"kubernetes.io/projected/ca33f358-fe61-4300-8b66-6aad28787b85-kube-api-access-zhk2f\") pod \"maas-api-779459d849-64v4d\" (UID: \"ca33f358-fe61-4300-8b66-6aad28787b85\") " pod="opendatahub/maas-api-779459d849-64v4d" Jun 02 19:12:33.668855 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:33.668818 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca33f358-fe61-4300-8b66-6aad28787b85-maas-api-tls\") pod \"maas-api-779459d849-64v4d\" (UID: \"ca33f358-fe61-4300-8b66-6aad28787b85\") " pod="opendatahub/maas-api-779459d849-64v4d" Jun 02 19:12:33.671259 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:33.671210 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca33f358-fe61-4300-8b66-6aad28787b85-maas-api-tls\") pod \"maas-api-779459d849-64v4d\" (UID: \"ca33f358-fe61-4300-8b66-6aad28787b85\") " pod="opendatahub/maas-api-779459d849-64v4d" Jun 02 19:12:33.897610 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:33.897580 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-779459d849-64v4d" Jun 02 19:12:34.023292 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:34.023263 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-779459d849-64v4d"] Jun 02 19:12:34.026509 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:12:34.026481 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca33f358_fe61_4300_8b66_6aad28787b85.slice/crio-66d61da4a0eee5ea08c190bc675d1e58622fe144bc097a68a5ad5103df88b150 WatchSource:0}: Error finding container 66d61da4a0eee5ea08c190bc675d1e58622fe144bc097a68a5ad5103df88b150: Status 404 returned error can't find the container with id 66d61da4a0eee5ea08c190bc675d1e58622fe144bc097a68a5ad5103df88b150 Jun 02 19:12:34.659916 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:34.659874 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-779459d849-64v4d" event={"ID":"ca33f358-fe61-4300-8b66-6aad28787b85","Type":"ContainerStarted","Data":"66d61da4a0eee5ea08c190bc675d1e58622fe144bc097a68a5ad5103df88b150"} Jun 02 19:12:36.669071 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:36.668999 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-779459d849-64v4d" event={"ID":"ca33f358-fe61-4300-8b66-6aad28787b85","Type":"ContainerStarted","Data":"e3e9b48ec00f4ce01a1340f2782b2a7b3ad59abdfca178ed05278f9511557d2d"} Jun 02 19:12:36.669429 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:36.669109 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-779459d849-64v4d" Jun 02 19:12:36.706265 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:36.706201 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-779459d849-64v4d" podStartSLOduration=2.335125208 podStartE2EDuration="4.70618923s" podCreationTimestamp="2026-06-02 19:12:32 +0000 UTC" firstStartedPulling="2026-06-02 19:12:34.027603523 +0000 UTC m=+516.845573225" lastFinishedPulling="2026-06-02 19:12:36.398667533 +0000 UTC m=+519.216637247" observedRunningTime="2026-06-02 19:12:36.70584691 +0000 UTC m=+519.523816635" watchObservedRunningTime="2026-06-02 19:12:36.70618923 +0000 UTC m=+519.524158954" Jun 02 19:12:42.678696 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:42.678644 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-779459d849-64v4d" Jun 02 19:12:44.566297 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:44.566259 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:12:44.566767 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:44.566534 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" containerID="cri-o://cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07" gracePeriod=30 Jun 02 19:12:46.206907 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.206884 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:46.372330 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.372259 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc78t\" (UniqueName: \"kubernetes.io/projected/7484c17d-2c0e-4684-89e1-8695190dde0a-kube-api-access-wc78t\") pod \"7484c17d-2c0e-4684-89e1-8695190dde0a\" (UID: \"7484c17d-2c0e-4684-89e1-8695190dde0a\") " Jun 02 19:12:46.374374 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.374338 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7484c17d-2c0e-4684-89e1-8695190dde0a-kube-api-access-wc78t" (OuterVolumeSpecName: "kube-api-access-wc78t") pod "7484c17d-2c0e-4684-89e1-8695190dde0a" (UID: "7484c17d-2c0e-4684-89e1-8695190dde0a"). InnerVolumeSpecName "kube-api-access-wc78t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:12:46.472876 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.472830 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc78t\" (UniqueName: \"kubernetes.io/projected/7484c17d-2c0e-4684-89e1-8695190dde0a-kube-api-access-wc78t\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:12:46.712285 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.712257 2574 generic.go:358] "Generic (PLEG): container finished" podID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerID="cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07" exitCode=143 Jun 02 19:12:46.712393 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.712323 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:46.712393 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.712345 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7484c17d-2c0e-4684-89e1-8695190dde0a","Type":"ContainerDied","Data":"cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07"} Jun 02 19:12:46.712393 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.712380 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7484c17d-2c0e-4684-89e1-8695190dde0a","Type":"ContainerDied","Data":"dcd74361fb1c3ab6a9f4c1ce6be6c3426802416901f2ccbfc427299a718edce6"} Jun 02 19:12:46.712508 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.712395 2574 scope.go:117] "RemoveContainer" containerID="cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07" Jun 02 19:12:46.721622 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.721608 2574 scope.go:117] "RemoveContainer" containerID="cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07" Jun 02 19:12:46.721871 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:12:46.721845 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07\": container with ID starting with cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07 not found: ID does not exist" containerID="cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07" Jun 02 19:12:46.721973 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.721872 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07"} err="failed to get container status \"cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07\": rpc error: code = NotFound desc = could not find container \"cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07\": container with ID starting with cfa38f7028bdd3e9488b789d6b002952288a55a28562cb1cc06ed0d55c93da07 not found: ID does not exist" Jun 02 19:12:46.732260 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.732236 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:12:46.736390 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.736373 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:12:46.757324 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.757307 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:12:46.757626 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.757615 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" Jun 02 19:12:46.757672 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.757627 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" Jun 02 19:12:46.757706 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.757696 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" containerName="keycloak" Jun 02 19:12:46.761858 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.761843 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:46.764185 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.764168 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 02 19:12:46.764185 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.764180 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 19:12:46.764352 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.764198 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 19:12:46.764352 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.764169 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 19:12:46.764352 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.764174 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-v9wdh\"" Jun 02 19:12:46.768654 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.768637 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:12:46.876863 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.876842 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/792dafa4-f5cb-4cab-9676-702161b0c2cd-test-realms\") pod \"maas-keycloak-0\" (UID: \"792dafa4-f5cb-4cab-9676-702161b0c2cd\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:46.876962 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.876879 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-749zt\" (UniqueName: \"kubernetes.io/projected/792dafa4-f5cb-4cab-9676-702161b0c2cd-kube-api-access-749zt\") pod \"maas-keycloak-0\" (UID: \"792dafa4-f5cb-4cab-9676-702161b0c2cd\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:46.977435 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.977378 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-749zt\" (UniqueName: \"kubernetes.io/projected/792dafa4-f5cb-4cab-9676-702161b0c2cd-kube-api-access-749zt\") pod \"maas-keycloak-0\" (UID: \"792dafa4-f5cb-4cab-9676-702161b0c2cd\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:46.977518 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.977471 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/792dafa4-f5cb-4cab-9676-702161b0c2cd-test-realms\") pod \"maas-keycloak-0\" (UID: \"792dafa4-f5cb-4cab-9676-702161b0c2cd\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:46.978126 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.978109 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/792dafa4-f5cb-4cab-9676-702161b0c2cd-test-realms\") pod \"maas-keycloak-0\" (UID: \"792dafa4-f5cb-4cab-9676-702161b0c2cd\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:46.985587 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:46.985568 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-749zt\" (UniqueName: \"kubernetes.io/projected/792dafa4-f5cb-4cab-9676-702161b0c2cd-kube-api-access-749zt\") pod \"maas-keycloak-0\" (UID: \"792dafa4-f5cb-4cab-9676-702161b0c2cd\") " pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:47.071936 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:47.071910 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:47.214800 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:47.214774 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 19:12:47.218003 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:12:47.217976 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod792dafa4_f5cb_4cab_9676_702161b0c2cd.slice/crio-f882a38b0ad6ee526b0db5cc207134c6a37d286245427c5b7004b8999eb81124 WatchSource:0}: Error finding container f882a38b0ad6ee526b0db5cc207134c6a37d286245427c5b7004b8999eb81124: Status 404 returned error can't find the container with id f882a38b0ad6ee526b0db5cc207134c6a37d286245427c5b7004b8999eb81124 Jun 02 19:12:47.683672 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:47.683647 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7484c17d-2c0e-4684-89e1-8695190dde0a" path="/var/lib/kubelet/pods/7484c17d-2c0e-4684-89e1-8695190dde0a/volumes" Jun 02 19:12:47.718245 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:47.718193 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"792dafa4-f5cb-4cab-9676-702161b0c2cd","Type":"ContainerStarted","Data":"f882a38b0ad6ee526b0db5cc207134c6a37d286245427c5b7004b8999eb81124"} Jun 02 19:12:48.723833 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:48.723795 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"792dafa4-f5cb-4cab-9676-702161b0c2cd","Type":"ContainerStarted","Data":"f7ca8f8daa03b47ddf2ee1c5504032697fdd0c8c8cb8b0742a43e44793e42b00"} Jun 02 19:12:48.741553 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:48.741453 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=2.304413212 podStartE2EDuration="2.741432968s" podCreationTimestamp="2026-06-02 19:12:46 +0000 UTC" firstStartedPulling="2026-06-02 19:12:47.219288338 +0000 UTC m=+530.037258040" lastFinishedPulling="2026-06-02 19:12:47.656308089 +0000 UTC m=+530.474277796" observedRunningTime="2026-06-02 19:12:48.739768733 +0000 UTC m=+531.557738451" watchObservedRunningTime="2026-06-02 19:12:48.741432968 +0000 UTC m=+531.559402697" Jun 02 19:12:49.072358 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:49.072254 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:49.073945 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:49.073913 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:12:50.073492 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:50.073439 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:12:51.072670 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:51.072617 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:12:52.072978 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:52.072931 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:12:53.072701 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:53.072654 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:12:54.073448 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:54.073033 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:12:55.072919 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:55.072870 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:12:56.072725 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:56.072677 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:12:57.072739 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:57.072701 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 19:12:57.073210 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:57.072982 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:12:58.072423 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:58.072378 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:12:59.073183 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:12:59.073136 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:13:00.072954 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:00.072904 2574 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.36:9000/health/started\": dial tcp 10.133.0.36:9000: connect: connection refused" Jun 02 19:13:01.206523 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:01.206489 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 19:13:01.225484 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:01.225439 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="792dafa4-f5cb-4cab-9676-702161b0c2cd" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 19:13:11.212380 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:11.212341 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 19:13:21.956933 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:21.956894 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84bcbbcff9-4km7v"] Jun 02 19:13:21.961075 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:21.961055 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84bcbbcff9-4km7v" Jun 02 19:13:21.963431 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:21.963408 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 02 19:13:21.967262 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:21.967232 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84bcbbcff9-4km7v"] Jun 02 19:13:22.101566 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.101545 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f2db34ca-7f1f-4f6e-8f98-779381f53f93-tls-cert\") pod \"authorino-84bcbbcff9-4km7v\" (UID: \"f2db34ca-7f1f-4f6e-8f98-779381f53f93\") " pod="kuadrant-system/authorino-84bcbbcff9-4km7v" Jun 02 19:13:22.101669 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.101574 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t94q6\" (UniqueName: \"kubernetes.io/projected/f2db34ca-7f1f-4f6e-8f98-779381f53f93-kube-api-access-t94q6\") pod \"authorino-84bcbbcff9-4km7v\" (UID: \"f2db34ca-7f1f-4f6e-8f98-779381f53f93\") " pod="kuadrant-system/authorino-84bcbbcff9-4km7v" Jun 02 19:13:22.101770 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.101741 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/f2db34ca-7f1f-4f6e-8f98-779381f53f93-oidc-ca\") pod \"authorino-84bcbbcff9-4km7v\" (UID: \"f2db34ca-7f1f-4f6e-8f98-779381f53f93\") " pod="kuadrant-system/authorino-84bcbbcff9-4km7v" Jun 02 19:13:22.202950 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.202921 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f2db34ca-7f1f-4f6e-8f98-779381f53f93-tls-cert\") pod \"authorino-84bcbbcff9-4km7v\" (UID: \"f2db34ca-7f1f-4f6e-8f98-779381f53f93\") " pod="kuadrant-system/authorino-84bcbbcff9-4km7v" Jun 02 19:13:22.203061 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.202954 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t94q6\" (UniqueName: \"kubernetes.io/projected/f2db34ca-7f1f-4f6e-8f98-779381f53f93-kube-api-access-t94q6\") pod \"authorino-84bcbbcff9-4km7v\" (UID: \"f2db34ca-7f1f-4f6e-8f98-779381f53f93\") " pod="kuadrant-system/authorino-84bcbbcff9-4km7v" Jun 02 19:13:22.203061 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.203020 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/f2db34ca-7f1f-4f6e-8f98-779381f53f93-oidc-ca\") pod \"authorino-84bcbbcff9-4km7v\" (UID: \"f2db34ca-7f1f-4f6e-8f98-779381f53f93\") " pod="kuadrant-system/authorino-84bcbbcff9-4km7v" Jun 02 19:13:22.203611 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.203593 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/f2db34ca-7f1f-4f6e-8f98-779381f53f93-oidc-ca\") pod \"authorino-84bcbbcff9-4km7v\" (UID: \"f2db34ca-7f1f-4f6e-8f98-779381f53f93\") " pod="kuadrant-system/authorino-84bcbbcff9-4km7v" Jun 02 19:13:22.205742 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.205720 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f2db34ca-7f1f-4f6e-8f98-779381f53f93-tls-cert\") pod \"authorino-84bcbbcff9-4km7v\" (UID: \"f2db34ca-7f1f-4f6e-8f98-779381f53f93\") " pod="kuadrant-system/authorino-84bcbbcff9-4km7v" Jun 02 19:13:22.210895 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.210828 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t94q6\" (UniqueName: \"kubernetes.io/projected/f2db34ca-7f1f-4f6e-8f98-779381f53f93-kube-api-access-t94q6\") pod \"authorino-84bcbbcff9-4km7v\" (UID: \"f2db34ca-7f1f-4f6e-8f98-779381f53f93\") " pod="kuadrant-system/authorino-84bcbbcff9-4km7v" Jun 02 19:13:22.273686 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.273665 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84bcbbcff9-4km7v" Jun 02 19:13:22.394870 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.394841 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84bcbbcff9-4km7v"] Jun 02 19:13:22.397086 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:13:22.397060 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2db34ca_7f1f_4f6e_8f98_779381f53f93.slice/crio-92f4160a5e9862f813149859f49b260b6d80bd3964337a8ada7a641ab69379aa WatchSource:0}: Error finding container 92f4160a5e9862f813149859f49b260b6d80bd3964337a8ada7a641ab69379aa: Status 404 returned error can't find the container with id 92f4160a5e9862f813149859f49b260b6d80bd3964337a8ada7a641ab69379aa Jun 02 19:13:22.857049 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.857012 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84bcbbcff9-4km7v" event={"ID":"f2db34ca-7f1f-4f6e-8f98-779381f53f93","Type":"ContainerStarted","Data":"eab5f566d821237bdd22c4f2fa469c138582fe56d984f54098ec3f52e2fb17f8"} Jun 02 19:13:22.857180 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.857065 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84bcbbcff9-4km7v" event={"ID":"f2db34ca-7f1f-4f6e-8f98-779381f53f93","Type":"ContainerStarted","Data":"92f4160a5e9862f813149859f49b260b6d80bd3964337a8ada7a641ab69379aa"} Jun 02 19:13:22.872491 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.872415 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-84bcbbcff9-4km7v" podStartSLOduration=1.513372328 podStartE2EDuration="1.872399571s" podCreationTimestamp="2026-06-02 19:13:21 +0000 UTC" firstStartedPulling="2026-06-02 19:13:22.398342566 +0000 UTC m=+565.216312268" lastFinishedPulling="2026-06-02 19:13:22.757369807 +0000 UTC m=+565.575339511" observedRunningTime="2026-06-02 19:13:22.870715866 +0000 UTC m=+565.688685614" watchObservedRunningTime="2026-06-02 19:13:22.872399571 +0000 UTC m=+565.690369295" Jun 02 19:13:22.900365 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.900324 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c69447669-8lr7j"] Jun 02 19:13:22.900588 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:22.900566 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-5c69447669-8lr7j" podUID="443df020-d00d-47df-8604-9c3d5bdfaeb1" containerName="authorino" containerID="cri-o://388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02" gracePeriod=30 Jun 02 19:13:23.141350 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.141267 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c69447669-8lr7j" Jun 02 19:13:23.311839 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.311752 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/443df020-d00d-47df-8604-9c3d5bdfaeb1-tls-cert\") pod \"443df020-d00d-47df-8604-9c3d5bdfaeb1\" (UID: \"443df020-d00d-47df-8604-9c3d5bdfaeb1\") " Jun 02 19:13:23.311839 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.311797 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7grd\" (UniqueName: \"kubernetes.io/projected/443df020-d00d-47df-8604-9c3d5bdfaeb1-kube-api-access-c7grd\") pod \"443df020-d00d-47df-8604-9c3d5bdfaeb1\" (UID: \"443df020-d00d-47df-8604-9c3d5bdfaeb1\") " Jun 02 19:13:23.315105 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.315054 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/443df020-d00d-47df-8604-9c3d5bdfaeb1-kube-api-access-c7grd" (OuterVolumeSpecName: "kube-api-access-c7grd") pod "443df020-d00d-47df-8604-9c3d5bdfaeb1" (UID: "443df020-d00d-47df-8604-9c3d5bdfaeb1"). InnerVolumeSpecName "kube-api-access-c7grd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 19:13:23.325173 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.325141 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/443df020-d00d-47df-8604-9c3d5bdfaeb1-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "443df020-d00d-47df-8604-9c3d5bdfaeb1" (UID: "443df020-d00d-47df-8604-9c3d5bdfaeb1"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 19:13:23.412805 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.412778 2574 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/443df020-d00d-47df-8604-9c3d5bdfaeb1-tls-cert\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:13:23.412805 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.412805 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7grd\" (UniqueName: \"kubernetes.io/projected/443df020-d00d-47df-8604-9c3d5bdfaeb1-kube-api-access-c7grd\") on node \"ip-10-0-138-102.ec2.internal\" DevicePath \"\"" Jun 02 19:13:23.860964 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.860936 2574 generic.go:358] "Generic (PLEG): container finished" podID="443df020-d00d-47df-8604-9c3d5bdfaeb1" containerID="388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02" exitCode=0 Jun 02 19:13:23.861121 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.860985 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c69447669-8lr7j" Jun 02 19:13:23.861121 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.861010 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c69447669-8lr7j" event={"ID":"443df020-d00d-47df-8604-9c3d5bdfaeb1","Type":"ContainerDied","Data":"388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02"} Jun 02 19:13:23.861121 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.861052 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c69447669-8lr7j" event={"ID":"443df020-d00d-47df-8604-9c3d5bdfaeb1","Type":"ContainerDied","Data":"b205546d207546795ca12c86f0fb52be1f28505e1f89c6ec4677816139388a2f"} Jun 02 19:13:23.861121 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.861072 2574 scope.go:117] "RemoveContainer" containerID="388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02" Jun 02 19:13:23.868763 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.868743 2574 scope.go:117] "RemoveContainer" containerID="388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02" Jun 02 19:13:23.869031 ip-10-0-138-102 kubenswrapper[2574]: E0602 19:13:23.869005 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02\": container with ID starting with 388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02 not found: ID does not exist" containerID="388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02" Jun 02 19:13:23.869122 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.869043 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02"} err="failed to get container status \"388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02\": rpc error: code = NotFound desc = could not find container \"388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02\": container with ID starting with 388d8e813bddf7d439c911216f285dd8a8fa55a180f148c9b5c97da02c52eb02 not found: ID does not exist" Jun 02 19:13:23.877059 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.877039 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c69447669-8lr7j"] Jun 02 19:13:23.883381 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:23.883362 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5c69447669-8lr7j"] Jun 02 19:13:25.683884 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:25.683841 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="443df020-d00d-47df-8604-9c3d5bdfaeb1" path="/var/lib/kubelet/pods/443df020-d00d-47df-8604-9c3d5bdfaeb1/volumes" Jun 02 19:13:53.371347 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.371310 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml"] Jun 02 19:13:53.371785 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.371648 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="443df020-d00d-47df-8604-9c3d5bdfaeb1" containerName="authorino" Jun 02 19:13:53.371785 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.371658 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="443df020-d00d-47df-8604-9c3d5bdfaeb1" containerName="authorino" Jun 02 19:13:53.371785 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.371720 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="443df020-d00d-47df-8604-9c3d5bdfaeb1" containerName="authorino" Jun 02 19:13:53.374938 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.374922 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.377867 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.377847 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 19:13:53.377970 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.377847 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 02 19:13:53.378551 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.378533 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 19:13:53.378618 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.378555 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qjz9q\"" Jun 02 19:13:53.398792 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.398769 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml"] Jun 02 19:13:53.433555 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.433535 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/759800b4-0cf5-4e7e-bbe0-600fdca80faa-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.433653 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.433578 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.433653 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.433606 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.433653 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.433627 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.433653 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.433647 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gzg6\" (UniqueName: \"kubernetes.io/projected/759800b4-0cf5-4e7e-bbe0-600fdca80faa-kube-api-access-6gzg6\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.433811 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.433709 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.534059 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.534034 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/759800b4-0cf5-4e7e-bbe0-600fdca80faa-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.534168 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.534077 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.534168 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.534097 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.534168 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.534124 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.534168 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.534150 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6gzg6\" (UniqueName: \"kubernetes.io/projected/759800b4-0cf5-4e7e-bbe0-600fdca80faa-kube-api-access-6gzg6\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.534383 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.534212 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.534612 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.534585 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.534717 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.534624 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.534717 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.534670 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.536308 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.536289 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/759800b4-0cf5-4e7e-bbe0-600fdca80faa-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.536400 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.536384 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/759800b4-0cf5-4e7e-bbe0-600fdca80faa-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.542467 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.542450 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gzg6\" (UniqueName: \"kubernetes.io/projected/759800b4-0cf5-4e7e-bbe0-600fdca80faa-kube-api-access-6gzg6\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml\" (UID: \"759800b4-0cf5-4e7e-bbe0-600fdca80faa\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.685949 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.685926 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:13:53.862165 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.862139 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml"] Jun 02 19:13:53.863268 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:13:53.863244 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759800b4_0cf5_4e7e_bbe0_600fdca80faa.slice/crio-73a7e2d776e406197e817bf6e84d0b9a3196a39d3a8a9ed07b38f93429ae094d WatchSource:0}: Error finding container 73a7e2d776e406197e817bf6e84d0b9a3196a39d3a8a9ed07b38f93429ae094d: Status 404 returned error can't find the container with id 73a7e2d776e406197e817bf6e84d0b9a3196a39d3a8a9ed07b38f93429ae094d Jun 02 19:13:53.964559 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:53.964500 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" event={"ID":"759800b4-0cf5-4e7e-bbe0-600fdca80faa","Type":"ContainerStarted","Data":"73a7e2d776e406197e817bf6e84d0b9a3196a39d3a8a9ed07b38f93429ae094d"} Jun 02 19:13:57.982279 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:57.982243 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" event={"ID":"759800b4-0cf5-4e7e-bbe0-600fdca80faa","Type":"ContainerStarted","Data":"dcc618f1c8280dedc62d574b7b8e8e7d3dcc783b87cd7b4bdbc3018e1d4d5182"} Jun 02 19:13:58.987084 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:58.987046 2574 generic.go:358] "Generic (PLEG): container finished" podID="759800b4-0cf5-4e7e-bbe0-600fdca80faa" containerID="dcc618f1c8280dedc62d574b7b8e8e7d3dcc783b87cd7b4bdbc3018e1d4d5182" exitCode=0 Jun 02 19:13:58.987381 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:13:58.987135 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" event={"ID":"759800b4-0cf5-4e7e-bbe0-600fdca80faa","Type":"ContainerDied","Data":"dcc618f1c8280dedc62d574b7b8e8e7d3dcc783b87cd7b4bdbc3018e1d4d5182"} Jun 02 19:14:00.996780 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:00.996749 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" event={"ID":"759800b4-0cf5-4e7e-bbe0-600fdca80faa","Type":"ContainerStarted","Data":"e6d06de2326d21d6bf5774b7cdef54a264fa81ee1c5e4b1741ee34238d530e85"} Jun 02 19:14:01.016433 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:01.016379 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" podStartSLOduration=1.5889067620000001 podStartE2EDuration="8.016362779s" podCreationTimestamp="2026-06-02 19:13:53 +0000 UTC" firstStartedPulling="2026-06-02 19:13:53.864922412 +0000 UTC m=+596.682892113" lastFinishedPulling="2026-06-02 19:14:00.292378417 +0000 UTC m=+603.110348130" observedRunningTime="2026-06-02 19:14:01.015108957 +0000 UTC m=+603.833078682" watchObservedRunningTime="2026-06-02 19:14:01.016362779 +0000 UTC m=+603.834332503" Jun 02 19:14:03.686767 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:03.686738 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:14:03.686767 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:03.686771 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:14:03.699068 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:03.699045 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:14:04.017281 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:04.017197 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml" Jun 02 19:14:24.058754 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.058668 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g"] Jun 02 19:14:24.062504 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.062479 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.064785 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.064767 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 02 19:14:24.071047 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.070889 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g"] Jun 02 19:14:24.171273 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.171250 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.171388 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.171286 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.171388 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.171302 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.171388 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.171350 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmlg\" (UniqueName: \"kubernetes.io/projected/06406876-db3c-463f-9339-77804526d3c5-kube-api-access-cbmlg\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.171492 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.171400 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.171492 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.171421 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06406876-db3c-463f-9339-77804526d3c5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.271835 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.271813 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.271941 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.271847 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.271941 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.271865 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.272011 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.271952 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmlg\" (UniqueName: \"kubernetes.io/projected/06406876-db3c-463f-9339-77804526d3c5-kube-api-access-cbmlg\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.272045 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.272021 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.272045 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.272042 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06406876-db3c-463f-9339-77804526d3c5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.272244 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.272202 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.272295 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.272257 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.272410 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.272394 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.274095 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.274074 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06406876-db3c-463f-9339-77804526d3c5-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.274357 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.274339 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06406876-db3c-463f-9339-77804526d3c5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.279629 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.279610 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmlg\" (UniqueName: \"kubernetes.io/projected/06406876-db3c-463f-9339-77804526d3c5-kube-api-access-cbmlg\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g\" (UID: \"06406876-db3c-463f-9339-77804526d3c5\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.374823 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.374774 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:24.502898 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:24.502873 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g"] Jun 02 19:14:24.505459 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:14:24.505433 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06406876_db3c_463f_9339_77804526d3c5.slice/crio-91f100f3d4b0d51723b5464703e1de817f65ba656b1af8eeb351591d4d4dc3b3 WatchSource:0}: Error finding container 91f100f3d4b0d51723b5464703e1de817f65ba656b1af8eeb351591d4d4dc3b3: Status 404 returned error can't find the container with id 91f100f3d4b0d51723b5464703e1de817f65ba656b1af8eeb351591d4d4dc3b3 Jun 02 19:14:25.080335 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:25.080296 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" event={"ID":"06406876-db3c-463f-9339-77804526d3c5","Type":"ContainerStarted","Data":"fa9d6f80ecb15f9bf5fa822e037ce45c18337929407614c6a8f668c212613d66"} Jun 02 19:14:25.080335 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:25.080338 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" event={"ID":"06406876-db3c-463f-9339-77804526d3c5","Type":"ContainerStarted","Data":"91f100f3d4b0d51723b5464703e1de817f65ba656b1af8eeb351591d4d4dc3b3"} Jun 02 19:14:26.085441 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:26.085406 2574 generic.go:358] "Generic (PLEG): container finished" podID="06406876-db3c-463f-9339-77804526d3c5" containerID="fa9d6f80ecb15f9bf5fa822e037ce45c18337929407614c6a8f668c212613d66" exitCode=0 Jun 02 19:14:26.085910 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:26.085498 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" event={"ID":"06406876-db3c-463f-9339-77804526d3c5","Type":"ContainerDied","Data":"fa9d6f80ecb15f9bf5fa822e037ce45c18337929407614c6a8f668c212613d66"} Jun 02 19:14:27.090698 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:27.090671 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" event={"ID":"06406876-db3c-463f-9339-77804526d3c5","Type":"ContainerStarted","Data":"526f87fc3062de14db2f78a618758e1c8491a52a814e4c2736689d8e4acba5df"} Jun 02 19:14:27.109695 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:27.109653 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" podStartSLOduration=2.685060017 podStartE2EDuration="3.109639537s" podCreationTimestamp="2026-06-02 19:14:24 +0000 UTC" firstStartedPulling="2026-06-02 19:14:26.086076871 +0000 UTC m=+628.904046573" lastFinishedPulling="2026-06-02 19:14:26.510656378 +0000 UTC m=+629.328626093" observedRunningTime="2026-06-02 19:14:27.108034965 +0000 UTC m=+629.926004689" watchObservedRunningTime="2026-06-02 19:14:27.109639537 +0000 UTC m=+629.927609309" Jun 02 19:14:31.086764 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.083878 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn"] Jun 02 19:14:31.106090 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.106045 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn"] Jun 02 19:14:31.106273 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.106170 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.109248 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.109227 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 02 19:14:31.226065 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.226032 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.226180 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.226120 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.226180 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.226147 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6367f81-6add-4408-bf1f-7ce08547200a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.226180 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.226166 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6kwp\" (UniqueName: \"kubernetes.io/projected/f6367f81-6add-4408-bf1f-7ce08547200a-kube-api-access-c6kwp\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.226350 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.226245 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.226350 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.226298 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.327608 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.327580 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6367f81-6add-4408-bf1f-7ce08547200a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.327729 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.327613 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c6kwp\" (UniqueName: \"kubernetes.io/projected/f6367f81-6add-4408-bf1f-7ce08547200a-kube-api-access-c6kwp\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.327729 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.327631 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.327729 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.327662 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.327729 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.327705 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.327942 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.327788 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.328061 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.328041 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.328120 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.328075 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.328155 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.328139 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.329987 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.329966 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6367f81-6add-4408-bf1f-7ce08547200a-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.330205 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.330184 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6367f81-6add-4408-bf1f-7ce08547200a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.336125 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.336101 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6kwp\" (UniqueName: \"kubernetes.io/projected/f6367f81-6add-4408-bf1f-7ce08547200a-kube-api-access-c6kwp\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn\" (UID: \"f6367f81-6add-4408-bf1f-7ce08547200a\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.416962 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.416937 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" Jun 02 19:14:31.545440 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:31.545416 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn"] Jun 02 19:14:31.547423 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:14:31.547391 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6367f81_6add_4408_bf1f_7ce08547200a.slice/crio-c95e7bd1d9dd926110c1e987de41b035b773ccbf37c30ab6a1d0e65eca5123a2 WatchSource:0}: Error finding container c95e7bd1d9dd926110c1e987de41b035b773ccbf37c30ab6a1d0e65eca5123a2: Status 404 returned error can't find the container with id c95e7bd1d9dd926110c1e987de41b035b773ccbf37c30ab6a1d0e65eca5123a2 Jun 02 19:14:32.109046 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:32.109012 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" event={"ID":"f6367f81-6add-4408-bf1f-7ce08547200a","Type":"ContainerStarted","Data":"314fec9268dc871bd740d9e194739bd4d2100938adabba99653017eaa7845a07"} Jun 02 19:14:32.109046 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:32.109045 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn" event={"ID":"f6367f81-6add-4408-bf1f-7ce08547200a","Type":"ContainerStarted","Data":"c95e7bd1d9dd926110c1e987de41b035b773ccbf37c30ab6a1d0e65eca5123a2"} Jun 02 19:14:34.375238 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:34.375195 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:34.375606 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:34.375250 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:34.387595 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:34.387574 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:14:35.133104 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:14:35.133076 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g" Jun 02 19:15:57.647316 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:15:57.647283 2574 scope.go:117] "RemoveContainer" containerID="2177afe4a2e6dda90f983b0a6f58166af6f4fa110c210a54c01f702754528b32" Jun 02 19:20:33.921905 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:33.921838 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn_f6367f81-6add-4408-bf1f-7ce08547200a/storage-initializer/0.log" Jun 02 19:20:35.023911 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:35.023881 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84bcbbcff9-4km7v_f2db34ca-7f1f-4f6e-8f98-779381f53f93/authorino/0.log" Jun 02 19:20:39.662782 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:39.662755 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-5d857d74c6-vg2b7_362fab38-9d9f-4d44-b8cd-b728ec0359ec/nginx/0.log" Jun 02 19:20:40.015280 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:40.015193 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-779459d849-64v4d_ca33f358-fe61-4300-8b66-6aad28787b85/maas-api/0.log" Jun 02 19:20:40.736113 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:40.736078 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69b56f574b-mrctf_f6aca53f-2c1f-41dc-9fd2-879c1a315b22/copy-manifests/0.log" Jun 02 19:20:40.743046 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:40.743024 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69b56f574b-mrctf_f6aca53f-2c1f-41dc-9fd2-879c1a315b22/manager/0.log" Jun 02 19:20:42.105866 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:42.105837 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84bcbbcff9-4km7v_f2db34ca-7f1f-4f6e-8f98-779381f53f93/authorino/0.log" Jun 02 19:20:42.440733 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:42.440710 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-jrd8w_d13a3fb7-eb6a-4e14-b984-612e04083bde/kuadrant-console-plugin/0.log" Jun 02 19:20:42.671030 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:42.670999 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-68fcb_528ae4cc-4828-44b4-981c-accde0a60458/manager/0.log" Jun 02 19:20:43.455202 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:43.455171 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f9d7d5cb6-7qqfr_9505c068-2097-4ab0-98da-648bcdca1989/kube-auth-proxy/0.log" Jun 02 19:20:44.373570 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:44.373540 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml_759800b4-0cf5-4e7e-bbe0-600fdca80faa/storage-initializer/0.log" Jun 02 19:20:44.380920 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:44.380898 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-6x6ml_759800b4-0cf5-4e7e-bbe0-600fdca80faa/main/0.log" Jun 02 19:20:44.739412 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:44.739381 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g_06406876-db3c-463f-9339-77804526d3c5/storage-initializer/0.log" Jun 02 19:20:44.746051 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:44.746000 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-7m78g_06406876-db3c-463f-9339-77804526d3c5/main/0.log" Jun 02 19:20:44.866845 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:44.866823 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-vhfbn_f6367f81-6add-4408-bf1f-7ce08547200a/storage-initializer/0.log" Jun 02 19:20:52.222484 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:52.222453 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b95tp_c641d72d-ffca-480e-9fa8-4b7200759721/global-pull-secret-syncer/0.log" Jun 02 19:20:52.379092 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:52.379065 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z4qbq_3e9cc9fb-40ad-4f89-923b-eaa811f2a424/konnectivity-agent/0.log" Jun 02 19:20:52.416431 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:52.416404 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-102.ec2.internal_9f5ba42d23e87cc5c7afced33d88e7c0/haproxy/0.log" Jun 02 19:20:56.852691 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:56.852664 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84bcbbcff9-4km7v_f2db34ca-7f1f-4f6e-8f98-779381f53f93/authorino/0.log" Jun 02 19:20:56.937811 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:56.937771 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-jrd8w_d13a3fb7-eb6a-4e14-b984-612e04083bde/kuadrant-console-plugin/0.log" Jun 02 19:20:57.018965 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:57.018940 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-68fcb_528ae4cc-4828-44b4-981c-accde0a60458/manager/0.log" Jun 02 19:20:58.640691 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:58.640667 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-6jsl7_febbc43c-abfd-4dee-99f6-4d33cdc29c37/kube-state-metrics/0.log" Jun 02 19:20:58.659561 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:58.659535 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-6jsl7_febbc43c-abfd-4dee-99f6-4d33cdc29c37/kube-rbac-proxy-main/0.log" Jun 02 19:20:58.678608 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:58.678582 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-6jsl7_febbc43c-abfd-4dee-99f6-4d33cdc29c37/kube-rbac-proxy-self/0.log" Jun 02 19:20:58.707099 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:58.707032 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6fd9645f5f-p58r4_10f50c19-0858-4245-a3d1-9ae7d69632ea/metrics-server/0.log" Jun 02 19:20:58.728780 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:58.728763 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-wcmhh_ab780fa2-b9b4-427f-a7a6-fe22af8479af/monitoring-plugin/0.log" Jun 02 19:20:58.899596 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:58.899579 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wkzf4_9dbf7bb9-a19a-4279-ae78-0e0129e976a7/node-exporter/0.log" Jun 02 19:20:58.922965 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:58.922943 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wkzf4_9dbf7bb9-a19a-4279-ae78-0e0129e976a7/kube-rbac-proxy/0.log" Jun 02 19:20:58.950766 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:58.950750 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wkzf4_9dbf7bb9-a19a-4279-ae78-0e0129e976a7/init-textfile/0.log" Jun 02 19:20:58.978605 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:58.978556 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-t5v5v_85509742-48ce-41fc-a98c-e8c35c87b828/kube-rbac-proxy-main/0.log" Jun 02 19:20:58.997448 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:58.997420 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-t5v5v_85509742-48ce-41fc-a98c-e8c35c87b828/kube-rbac-proxy-self/0.log" Jun 02 19:20:59.018766 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.018745 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-t5v5v_85509742-48ce-41fc-a98c-e8c35c87b828/openshift-state-metrics/0.log" Jun 02 19:20:59.074163 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.074149 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e3e1ad6e-7c1d-4121-b9e9-1bb659453659/prometheus/0.log" Jun 02 19:20:59.099194 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.099177 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e3e1ad6e-7c1d-4121-b9e9-1bb659453659/config-reloader/0.log" Jun 02 19:20:59.126414 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.126380 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e3e1ad6e-7c1d-4121-b9e9-1bb659453659/thanos-sidecar/0.log" Jun 02 19:20:59.154546 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.154529 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e3e1ad6e-7c1d-4121-b9e9-1bb659453659/kube-rbac-proxy-web/0.log" Jun 02 19:20:59.179109 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.179091 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e3e1ad6e-7c1d-4121-b9e9-1bb659453659/kube-rbac-proxy/0.log" Jun 02 19:20:59.202004 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.201984 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e3e1ad6e-7c1d-4121-b9e9-1bb659453659/kube-rbac-proxy-thanos/0.log" Jun 02 19:20:59.221857 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.221841 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e3e1ad6e-7c1d-4121-b9e9-1bb659453659/init-config-reloader/0.log" Jun 02 19:20:59.320104 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.320047 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747f495d59-8xkqf_5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c/telemeter-client/0.log" Jun 02 19:20:59.341211 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.341189 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747f495d59-8xkqf_5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c/reload/0.log" Jun 02 19:20:59.377608 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.377590 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-747f495d59-8xkqf_5b1ee5ef-21d7-438a-b3d5-e8ed0c5f124c/kube-rbac-proxy/0.log" Jun 02 19:20:59.409355 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.409339 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/thanos-query/0.log" Jun 02 19:20:59.431995 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.431968 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/kube-rbac-proxy-web/0.log" Jun 02 19:20:59.454012 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.453993 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/kube-rbac-proxy/0.log" Jun 02 19:20:59.476853 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.476833 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/prom-label-proxy/0.log" Jun 02 19:20:59.498474 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.498457 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/kube-rbac-proxy-rules/0.log" Jun 02 19:20:59.555813 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:20:59.555790 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-689cf468df-sh2kv_328db247-e97e-4842-85b2-073684347027/kube-rbac-proxy-metrics/0.log" Jun 02 19:21:01.551281 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.551243 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx"] Jun 02 19:21:01.554794 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.554774 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.556921 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.556902 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jdvqf\"/\"default-dockercfg-c2gzr\"" Jun 02 19:21:01.557576 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.557556 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdvqf\"/\"kube-root-ca.crt\"" Jun 02 19:21:01.557687 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.557576 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdvqf\"/\"openshift-service-ca.crt\"" Jun 02 19:21:01.561048 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.561026 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx"] Jun 02 19:21:01.654291 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.654265 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-sys\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.654418 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.654296 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-podres\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.654418 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.654317 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-lib-modules\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.654418 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.654336 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mqbg\" (UniqueName: \"kubernetes.io/projected/7366bed7-3e0c-4913-a341-634a7517e0ba-kube-api-access-7mqbg\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.654575 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.654475 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-proc\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.670956 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.670938 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-6q4kq_e25f59c3-6117-4dbb-8dc5-dd26b49d43f8/download-server/0.log" Jun 02 19:21:01.755077 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.755050 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-proc\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.755171 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.755100 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-sys\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.755171 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.755117 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-podres\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.755171 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.755137 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-lib-modules\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.755171 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.755156 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7mqbg\" (UniqueName: \"kubernetes.io/projected/7366bed7-3e0c-4913-a341-634a7517e0ba-kube-api-access-7mqbg\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.755171 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.755159 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-proc\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.755506 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.755274 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-sys\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.755506 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.755323 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-lib-modules\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.755506 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.755327 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7366bed7-3e0c-4913-a341-634a7517e0ba-podres\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.762715 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.762695 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mqbg\" (UniqueName: \"kubernetes.io/projected/7366bed7-3e0c-4913-a341-634a7517e0ba-kube-api-access-7mqbg\") pod \"perf-node-gather-daemonset-nn4nx\" (UID: \"7366bed7-3e0c-4913-a341-634a7517e0ba\") " pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.866894 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.866840 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:01.994772 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.994746 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx"] Jun 02 19:21:01.997362 ip-10-0-138-102 kubenswrapper[2574]: W0602 19:21:01.997332 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7366bed7_3e0c_4913_a341_634a7517e0ba.slice/crio-c3ba2479f64e7d4c5707b14d8ab091f8594becb57d0c5229ac07588be97c307f WatchSource:0}: Error finding container c3ba2479f64e7d4c5707b14d8ab091f8594becb57d0c5229ac07588be97c307f: Status 404 returned error can't find the container with id c3ba2479f64e7d4c5707b14d8ab091f8594becb57d0c5229ac07588be97c307f Jun 02 19:21:01.999324 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:01.999307 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 19:21:02.444285 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:02.444260 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" event={"ID":"7366bed7-3e0c-4913-a341-634a7517e0ba","Type":"ContainerStarted","Data":"ee00d4f3c47a8af7d30f0760a4b3b8b0b2ba0c3cb76f656514555d9934d459cb"} Jun 02 19:21:02.444433 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:02.444290 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" event={"ID":"7366bed7-3e0c-4913-a341-634a7517e0ba","Type":"ContainerStarted","Data":"c3ba2479f64e7d4c5707b14d8ab091f8594becb57d0c5229ac07588be97c307f"} Jun 02 19:21:02.444433 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:02.444313 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:02.461286 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:02.461238 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" podStartSLOduration=1.461204345 podStartE2EDuration="1.461204345s" podCreationTimestamp="2026-06-02 19:21:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 19:21:02.459357897 +0000 UTC m=+1025.277327621" watchObservedRunningTime="2026-06-02 19:21:02.461204345 +0000 UTC m=+1025.279174070" Jun 02 19:21:03.091861 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:03.091835 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-26xk6_5494ce57-6970-4bd2-aae6-6c83780cbc49/dns/0.log" Jun 02 19:21:03.112644 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:03.112612 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-26xk6_5494ce57-6970-4bd2-aae6-6c83780cbc49/kube-rbac-proxy/0.log" Jun 02 19:21:03.249780 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:03.249755 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dnhhf_6f947012-134a-4905-8242-57a12df7ec9f/dns-node-resolver/0.log" Jun 02 19:21:03.706436 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:03.706407 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-f7fdc8998-tqs6k_624f5b44-0801-4a57-bb6b-f22c6bcd2afd/registry/0.log" Jun 02 19:21:03.769717 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:03.769694 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wrs9f_9efd8fa3-600f-42a8-83f5-25d80caacdd7/node-ca/0.log" Jun 02 19:21:04.643272 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:04.643245 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f9d7d5cb6-7qqfr_9505c068-2097-4ab0-98da-648bcdca1989/kube-auth-proxy/0.log" Jun 02 19:21:05.280614 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:05.280592 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mv67v_5fdac944-f6ec-4d0d-8eac-31d0f8b3c5af/serve-healthcheck-canary/0.log" Jun 02 19:21:05.766238 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:05.766196 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2lskv_456f7870-5977-4ee2-8d13-2ae8a34f3119/kube-rbac-proxy/0.log" Jun 02 19:21:05.786020 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:05.785999 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2lskv_456f7870-5977-4ee2-8d13-2ae8a34f3119/exporter/0.log" Jun 02 19:21:05.806073 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:05.806052 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2lskv_456f7870-5977-4ee2-8d13-2ae8a34f3119/extractor/0.log" Jun 02 19:21:07.879548 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:07.879521 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-5d857d74c6-vg2b7_362fab38-9d9f-4d44-b8cd-b728ec0359ec/nginx/0.log" Jun 02 19:21:07.972003 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:07.971954 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-779459d849-64v4d_ca33f358-fe61-4300-8b66-6aad28787b85/maas-api/0.log" Jun 02 19:21:08.286846 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:08.286819 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69b56f574b-mrctf_f6aca53f-2c1f-41dc-9fd2-879c1a315b22/manager/0.log" Jun 02 19:21:08.305352 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:08.305335 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69b56f574b-mrctf_f6aca53f-2c1f-41dc-9fd2-879c1a315b22/copy-manifests/0.log" Jun 02 19:21:08.458063 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:08.458039 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jdvqf/perf-node-gather-daemonset-nn4nx" Jun 02 19:21:09.758768 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:09.758745 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6cd7cc9b94-lnxbr_b04d2724-3ba4-45bf-9784-daa2858ce165/manager/0.log" Jun 02 19:21:16.351505 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:16.351480 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2q4_ee8b07c2-5feb-4d66-87aa-aee5c65834dd/kube-multus-additional-cni-plugins/0.log" Jun 02 19:21:16.374008 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:16.373987 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2q4_ee8b07c2-5feb-4d66-87aa-aee5c65834dd/egress-router-binary-copy/0.log" Jun 02 19:21:16.392973 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:16.392951 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2q4_ee8b07c2-5feb-4d66-87aa-aee5c65834dd/cni-plugins/0.log" Jun 02 19:21:16.412683 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:16.412663 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2q4_ee8b07c2-5feb-4d66-87aa-aee5c65834dd/bond-cni-plugin/0.log" Jun 02 19:21:16.435792 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:16.435770 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2q4_ee8b07c2-5feb-4d66-87aa-aee5c65834dd/routeoverride-cni/0.log" Jun 02 19:21:16.456088 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:16.456064 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2q4_ee8b07c2-5feb-4d66-87aa-aee5c65834dd/whereabouts-cni-bincopy/0.log" Jun 02 19:21:16.480951 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:16.480920 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw2q4_ee8b07c2-5feb-4d66-87aa-aee5c65834dd/whereabouts-cni/0.log" Jun 02 19:21:16.571392 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:16.571359 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zvtrb_56b68433-83da-422d-be07-0a4e5c2e1f40/kube-multus/0.log" Jun 02 19:21:16.633286 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:16.633209 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mp2fh_9e006346-8709-4a02-bf44-d9e5a59bb74b/network-metrics-daemon/0.log" Jun 02 19:21:16.659958 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:16.659937 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mp2fh_9e006346-8709-4a02-bf44-d9e5a59bb74b/kube-rbac-proxy/0.log" Jun 02 19:21:17.533038 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:17.533001 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4mqnl_12107e6a-8aed-4528-a741-e28f41659026/ovn-controller/0.log" Jun 02 19:21:17.555018 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:17.554993 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4mqnl_12107e6a-8aed-4528-a741-e28f41659026/ovn-acl-logging/0.log" Jun 02 19:21:17.570708 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:17.570685 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4mqnl_12107e6a-8aed-4528-a741-e28f41659026/kube-rbac-proxy-node/0.log" Jun 02 19:21:17.597126 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:17.597109 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4mqnl_12107e6a-8aed-4528-a741-e28f41659026/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 19:21:17.616881 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:17.616858 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4mqnl_12107e6a-8aed-4528-a741-e28f41659026/northd/0.log" Jun 02 19:21:17.636936 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:17.636918 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4mqnl_12107e6a-8aed-4528-a741-e28f41659026/nbdb/0.log" Jun 02 19:21:17.656125 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:17.656104 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4mqnl_12107e6a-8aed-4528-a741-e28f41659026/sbdb/0.log" Jun 02 19:21:17.756985 ip-10-0-138-102 kubenswrapper[2574]: I0602 19:21:17.756963 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4mqnl_12107e6a-8aed-4528-a741-e28f41659026/ovnkube-controller/0.log"