May 28 20:52:29.761407 ip-10-0-140-94 systemd[1]: Starting Kubernetes Kubelet... May 28 20:52:30.237951 ip-10-0-140-94 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 20:52:30.237951 ip-10-0-140-94 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 28 20:52:30.237951 ip-10-0-140-94 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 20:52:30.237951 ip-10-0-140-94 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 28 20:52:30.237951 ip-10-0-140-94 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 20:52:30.241586 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.241492 2563 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 28 20:52:30.246326 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246306 2563 feature_gate.go:328] unrecognized feature gate: Example2 May 28 20:52:30.246326 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246322 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 20:52:30.246326 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246328 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 20:52:30.246326 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246332 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246337 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246341 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246345 2563 feature_gate.go:328] unrecognized feature gate: Example May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246348 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246352 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246356 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246361 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246366 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246370 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246373 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246377 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246381 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246385 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246389 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246392 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246397 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246400 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246404 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246409 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 20:52:30.246553 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246413 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246416 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246421 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246425 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246431 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246436 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246442 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246448 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246453 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246457 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246462 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246466 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246470 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246474 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246479 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246483 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246490 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246494 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246498 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 20:52:30.247401 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246503 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246507 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246512 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246516 2563 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246520 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246524 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246529 2563 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246533 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246537 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246541 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246546 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246549 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246553 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246557 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246561 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246565 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246569 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246573 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246577 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246581 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 20:52:30.248035 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246586 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246590 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246594 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246598 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246603 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246607 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246611 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246616 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246620 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246626 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246630 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246634 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246639 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246643 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246648 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246653 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246657 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246661 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246665 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246669 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 20:52:30.248540 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246673 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246678 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246682 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.246686 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247312 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247321 2563 feature_gate.go:328] unrecognized feature gate: Example2 May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247325 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247329 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247334 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247338 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247342 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247347 2563 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247351 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247355 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247359 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247363 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247368 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247373 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247377 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247382 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 20:52:30.249032 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247387 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247392 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247396 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247400 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247404 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247408 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247413 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247417 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247421 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247426 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247430 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247435 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247439 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247443 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247449 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247455 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247460 2563 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247464 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247469 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 20:52:30.249851 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247473 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247477 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247484 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247489 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247494 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247498 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247502 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247506 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247510 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247515 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247519 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247523 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247529 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247533 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247538 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247542 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247546 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247550 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247554 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 20:52:30.250716 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247558 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247564 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247568 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247572 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247577 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247581 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247585 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247589 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247593 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247597 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247601 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247605 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247609 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247614 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247618 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247622 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247627 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247631 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247635 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247639 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 20:52:30.251273 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247643 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247647 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247652 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247656 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247660 2563 feature_gate.go:328] unrecognized feature gate: Example May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247665 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247669 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247673 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247677 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247681 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247686 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.247690 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249095 2563 flags.go:64] FLAG: --address="0.0.0.0" May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249111 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249124 2563 flags.go:64] FLAG: --anonymous-auth="true" May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249130 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249137 2563 flags.go:64] FLAG: --authentication-token-webhook="false" May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249142 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249150 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249164 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249186 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 28 20:52:30.251849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249202 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249208 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249214 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249219 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249224 2563 flags.go:64] FLAG: --cgroup-root="" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249229 2563 flags.go:64] FLAG: --cgroups-per-qos="true" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249234 2563 flags.go:64] FLAG: --client-ca-file="" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249238 2563 flags.go:64] FLAG: --cloud-config="" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249243 2563 flags.go:64] FLAG: --cloud-provider="external" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249249 2563 flags.go:64] FLAG: --cluster-dns="[]" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249293 2563 flags.go:64] FLAG: --cluster-domain="" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249299 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249305 2563 flags.go:64] FLAG: --config-dir="" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249310 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249316 2563 flags.go:64] FLAG: --container-log-max-files="5" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249323 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249329 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249334 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249340 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249345 2563 flags.go:64] FLAG: --contention-profiling="false" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249350 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249355 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249360 2563 flags.go:64] FLAG: --cpu-manager-policy="none" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249365 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249372 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 28 20:52:30.252438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249377 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249382 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249387 2563 flags.go:64] FLAG: --enable-load-reader="false" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249392 2563 flags.go:64] FLAG: --enable-server="true" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249397 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249404 2563 flags.go:64] FLAG: --event-burst="100" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249409 2563 flags.go:64] FLAG: --event-qps="50" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249414 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249420 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249425 2563 flags.go:64] FLAG: --eviction-hard="" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249431 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249436 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249441 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249446 2563 flags.go:64] FLAG: --eviction-soft="" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249451 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249455 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249461 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249466 2563 flags.go:64] FLAG: --experimental-mounter-path="" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249471 2563 flags.go:64] FLAG: --fail-cgroupv1="false" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249476 2563 flags.go:64] FLAG: --fail-swap-on="true" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249481 2563 flags.go:64] FLAG: --feature-gates="" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249487 2563 flags.go:64] FLAG: --file-check-frequency="20s" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249492 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249498 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249503 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249508 2563 flags.go:64] FLAG: --healthz-port="10248" May 28 20:52:30.253062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249513 2563 flags.go:64] FLAG: --help="false" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249518 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-140-94.ec2.internal" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249523 2563 flags.go:64] FLAG: --housekeeping-interval="10s" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249528 2563 flags.go:64] FLAG: --http-check-frequency="20s" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249534 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249539 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249545 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249550 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249556 2563 flags.go:64] FLAG: --image-service-endpoint="" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249560 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249565 2563 flags.go:64] FLAG: --kube-api-burst="100" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249570 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249576 2563 flags.go:64] FLAG: --kube-api-qps="50" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249581 2563 flags.go:64] FLAG: --kube-reserved="" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249586 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249590 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249595 2563 flags.go:64] FLAG: --kubelet-cgroups="" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249600 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249605 2563 flags.go:64] FLAG: --lock-file="" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249610 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249615 2563 flags.go:64] FLAG: --log-flush-frequency="5s" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249620 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249629 2563 flags.go:64] FLAG: --log-json-split-stream="false" May 28 20:52:30.253709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249634 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249639 2563 flags.go:64] FLAG: --log-text-split-stream="false" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249644 2563 flags.go:64] FLAG: --logging-format="text" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249650 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249655 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249660 2563 flags.go:64] FLAG: --manifest-url="" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249665 2563 flags.go:64] FLAG: --manifest-url-header="" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249672 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249677 2563 flags.go:64] FLAG: --max-open-files="1000000" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249683 2563 flags.go:64] FLAG: --max-pods="110" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249689 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249694 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249699 2563 flags.go:64] FLAG: --memory-manager-policy="None" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249704 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249709 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249714 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249719 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249730 2563 flags.go:64] FLAG: --node-status-max-images="50" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249736 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249741 2563 flags.go:64] FLAG: --oom-score-adj="-999" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249746 2563 flags.go:64] FLAG: --pod-cidr="" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249751 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249759 2563 flags.go:64] FLAG: --pod-manifest-path="" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249764 2563 flags.go:64] FLAG: --pod-max-pids="-1" May 28 20:52:30.254282 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249769 2563 flags.go:64] FLAG: --pods-per-core="0" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249774 2563 flags.go:64] FLAG: --port="10250" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249779 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249784 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d9247c04bbe2a54e" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249789 2563 flags.go:64] FLAG: --qos-reserved="" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249794 2563 flags.go:64] FLAG: --read-only-port="10255" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249799 2563 flags.go:64] FLAG: --register-node="true" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249804 2563 flags.go:64] FLAG: --register-schedulable="true" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249809 2563 flags.go:64] FLAG: --register-with-taints="" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249815 2563 flags.go:64] FLAG: --registry-burst="10" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249820 2563 flags.go:64] FLAG: --registry-qps="5" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249826 2563 flags.go:64] FLAG: --reserved-cpus="" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249831 2563 flags.go:64] FLAG: --reserved-memory="" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249837 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249842 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249847 2563 flags.go:64] FLAG: --rotate-certificates="false" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249851 2563 flags.go:64] FLAG: --rotate-server-certificates="false" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249856 2563 flags.go:64] FLAG: --runonce="false" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249861 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249866 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249871 2563 flags.go:64] FLAG: --seccomp-default="false" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249875 2563 flags.go:64] FLAG: --serialize-image-pulls="true" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249880 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249886 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249891 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249897 2563 flags.go:64] FLAG: --storage-driver-password="root" May 28 20:52:30.255164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249901 2563 flags.go:64] FLAG: --storage-driver-secure="false" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249907 2563 flags.go:64] FLAG: --storage-driver-table="stats" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249912 2563 flags.go:64] FLAG: --storage-driver-user="root" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249917 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249922 2563 flags.go:64] FLAG: --sync-frequency="1m0s" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249927 2563 flags.go:64] FLAG: --system-cgroups="" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249931 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249941 2563 flags.go:64] FLAG: --system-reserved-cgroup="" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249946 2563 flags.go:64] FLAG: --tls-cert-file="" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249950 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249958 2563 flags.go:64] FLAG: --tls-min-version="" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249962 2563 flags.go:64] FLAG: --tls-private-key-file="" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249971 2563 flags.go:64] FLAG: --topology-manager-policy="none" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249976 2563 flags.go:64] FLAG: --topology-manager-policy-options="" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249981 2563 flags.go:64] FLAG: --topology-manager-scope="container" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249986 2563 flags.go:64] FLAG: --v="2" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249992 2563 flags.go:64] FLAG: --version="false" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.249999 2563 flags.go:64] FLAG: --vmodule="" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.250006 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.250012 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250165 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250189 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250195 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250200 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 20:52:30.256006 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250205 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250209 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250213 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250218 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250222 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250226 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250230 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250234 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250238 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250242 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250247 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250251 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250256 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250260 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250264 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250268 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250273 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250277 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250281 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 20:52:30.256653 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250288 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250301 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250306 2563 feature_gate.go:328] unrecognized feature gate: Example2 May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250311 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250317 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250321 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250326 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250330 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250334 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250338 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250343 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250348 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250352 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250357 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250361 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250365 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250370 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250374 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250379 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250384 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 20:52:30.257139 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250388 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250392 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250397 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250402 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250407 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250412 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250416 2563 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250420 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250425 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250429 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250433 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250437 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250442 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250447 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250451 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250455 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250460 2563 feature_gate.go:328] unrecognized feature gate: Example May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250465 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250471 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250477 2563 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 20:52:30.257650 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250482 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250487 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250491 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250495 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250500 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250504 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250508 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250512 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250516 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250520 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250524 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250528 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250532 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250536 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250540 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250544 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250548 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250553 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250558 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250562 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 20:52:30.258258 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250566 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 20:52:30.258761 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250570 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 20:52:30.258761 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.250574 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 20:52:30.258761 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.251115 2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 20:52:30.259578 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.259559 2563 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 28 20:52:30.259612 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.259580 2563 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 28 20:52:30.259645 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259628 2563 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 20:52:30.259645 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259634 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 20:52:30.259645 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259638 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 20:52:30.259645 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259643 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 20:52:30.259645 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259646 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259649 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259652 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259655 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259658 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259661 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259664 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259666 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259669 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259671 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259674 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259677 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259679 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259682 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259684 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259687 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259690 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259692 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259694 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259697 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 20:52:30.259777 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259704 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259707 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259709 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259712 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259714 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259717 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259720 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259722 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259724 2563 feature_gate.go:328] unrecognized feature gate: Example May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259727 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259730 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259732 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259736 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259740 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259745 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259748 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259750 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259753 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259756 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 20:52:30.260287 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259758 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259761 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259764 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259767 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259769 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259772 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259775 2563 feature_gate.go:328] unrecognized feature gate: Example2 May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259777 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259780 2563 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259783 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259786 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259789 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259791 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259794 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259797 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259799 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259801 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259804 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259807 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259809 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 20:52:30.260765 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259812 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259814 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259817 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259819 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259822 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259824 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259826 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259830 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259833 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259836 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259839 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259841 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259844 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259847 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259850 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259853 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259855 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259857 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259860 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259862 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 20:52:30.261267 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259865 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259867 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259870 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.259874 2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259986 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259992 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259995 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.259998 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260001 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260004 2563 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260007 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260010 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260012 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260015 2563 feature_gate.go:328] unrecognized feature gate: Example May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260017 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260020 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 20:52:30.261762 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260023 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260025 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260028 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260031 2563 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260034 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260036 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260039 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260041 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260044 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260046 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260049 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260052 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260054 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260057 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260059 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260062 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260064 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260067 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260069 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260072 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 20:52:30.262159 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260074 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260077 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260080 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260082 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260085 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260088 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260090 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260092 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260095 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260098 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260100 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260102 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260105 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260107 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260110 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260113 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260116 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260118 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260121 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260124 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 20:52:30.262789 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260126 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260128 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260131 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260134 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260137 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260140 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260142 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260145 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260147 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260149 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260152 2563 feature_gate.go:328] unrecognized feature gate: Example2 May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260155 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260157 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260159 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260162 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260164 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260167 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260181 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260183 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 20:52:30.263304 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260186 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260188 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260191 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260194 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260196 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260200 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260204 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260207 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260210 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260213 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260215 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260218 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260222 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260226 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:30.260229 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 20:52:30.263758 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.260234 2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 20:52:30.264156 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.261366 2563 server.go:962] "Client rotation is on, will bootstrap in background" May 28 20:52:30.266361 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.266347 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 28 20:52:30.267391 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.267379 2563 server.go:1019] "Starting client certificate rotation" May 28 20:52:30.267499 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.267485 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 20:52:30.267529 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.267517 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 20:52:30.297291 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.297275 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 20:52:30.299164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.299149 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 20:52:30.317917 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.317900 2563 log.go:25] "Validated CRI v1 runtime API" May 28 20:52:30.324144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.324129 2563 log.go:25] "Validated CRI v1 image API" May 28 20:52:30.325355 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.325338 2563 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 28 20:52:30.328815 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.328795 2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 cbb2dd21-db49-439c-b057-610604a599e8:/dev/nvme0n1p3 ef7a339e-a90b-463a-908e-9d6368c7d331:/dev/nvme0n1p4] May 28 20:52:30.328885 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.328813 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 28 20:52:30.334290 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.334156 2563 manager.go:217] Machine: {Timestamp:2026-05-28 20:52:30.333640937 +0000 UTC m=+0.451034513 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101029 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec248bd9630bdac74349b61cd02328b5 SystemUUID:ec248bd9-630b-dac7-4349-b61cd02328b5 BootID:c6204111-d902-486f-b1d2-9076fac21fdb Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:92:db:5d:84:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:92:db:5d:84:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:52:f9:3b:d6:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 28 20:52:30.335295 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.335283 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 28 20:52:30.335387 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.335375 2563 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 28 20:52:30.339197 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.339160 2563 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 28 20:52:30.339334 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.339200 2563 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-94.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 28 20:52:30.339383 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.339344 2563 topology_manager.go:138] "Creating topology manager with none policy" May 28 20:52:30.339383 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.339353 2563 container_manager_linux.go:306] "Creating device plugin manager" May 28 20:52:30.339383 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.339361 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 20:52:30.341012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.341001 2563 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 20:52:30.342268 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.342256 2563 state_mem.go:36] "Initialized new in-memory state store" May 28 20:52:30.342378 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.342369 2563 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 28 20:52:30.345943 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.345932 2563 kubelet.go:491] "Attempting to sync node with API server" May 28 20:52:30.345983 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.345947 2563 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 28 20:52:30.345983 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.345959 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 28 20:52:30.345983 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.345968 2563 kubelet.go:397] "Adding apiserver pod source" May 28 20:52:30.345983 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.345976 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 28 20:52:30.348313 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.348299 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 20:52:30.348370 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.348318 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 20:52:30.352311 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.352296 2563 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 28 20:52:30.353540 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.353522 2563 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 20:52:30.354259 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.354235 2563 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 28 20:52:30.355167 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355156 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 28 20:52:30.355223 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355187 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 28 20:52:30.355223 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355193 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 28 20:52:30.355223 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355199 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 28 20:52:30.355223 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355207 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 28 20:52:30.355223 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355213 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 28 20:52:30.355223 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355218 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 28 20:52:30.355223 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355223 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 28 20:52:30.355405 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355230 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 28 20:52:30.355405 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355236 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 28 20:52:30.355405 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355244 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 28 20:52:30.355511 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.355502 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 28 20:52:30.356536 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.356524 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 28 20:52:30.356588 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.356538 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 28 20:52:30.359788 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.359774 2563 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 28 20:52:30.359852 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.359810 2563 server.go:1295] "Started kubelet" May 28 20:52:30.359952 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.359899 2563 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 28 20:52:30.360068 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.359966 2563 server_v1.go:47] "podresources" method="list" useActivePods=true May 28 20:52:30.360068 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.359955 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 28 20:52:30.360589 ip-10-0-140-94 systemd[1]: Started Kubernetes Kubelet. May 28 20:52:30.361218 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.360984 2563 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 28 20:52:30.361218 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.361024 2563 server.go:317] "Adding debug handlers to kubelet server" May 28 20:52:30.366138 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.366121 2563 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-94.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 28 20:52:30.366321 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.366298 2563 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-94.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 28 20:52:30.366365 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.366325 2563 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 28 20:52:30.366822 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.366801 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 28 20:52:30.367408 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.367387 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 28 20:52:30.367722 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.367703 2563 volume_manager.go:295] "The desired_state_of_world populator starts" May 28 20:52:30.367722 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.367720 2563 volume_manager.go:297] "Starting Kubelet Volume Manager" May 28 20:52:30.367874 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.367817 2563 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 28 20:52:30.367931 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.367879 2563 reconstruct.go:97] "Volume reconstruction finished" May 28 20:52:30.367931 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.367887 2563 reconciler.go:26] "Reconciler: start to sync state" May 28 20:52:30.368671 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.368656 2563 factory.go:55] Registering systemd factory May 28 20:52:30.368671 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.368673 2563 factory.go:223] Registration of the systemd container factory successfully May 28 20:52:30.369336 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.368895 2563 factory.go:153] Registering CRI-O factory May 28 20:52:30.369336 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.368911 2563 factory.go:223] Registration of the crio container factory successfully May 28 20:52:30.369336 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.368919 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:30.369336 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.369046 2563 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 28 20:52:30.369559 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.368959 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 28 20:52:30.369559 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.369496 2563 factory.go:103] Registering Raw factory May 28 20:52:30.369559 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.369513 2563 manager.go:1196] Started watching for new ooms in manager May 28 20:52:30.370412 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.370394 2563 manager.go:319] Starting recovery of all containers May 28 20:52:30.371202 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.371161 2563 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" May 28 20:52:30.371305 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.371283 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-94.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" May 28 20:52:30.374793 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.370803 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-94.ec2.internal.18b3d5faacd6f781 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-94.ec2.internal,UID:ip-10-0-140-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-94.ec2.internal,},FirstTimestamp:2026-05-28 20:52:30.359787393 +0000 UTC m=+0.477180970,LastTimestamp:2026-05-28 20:52:30.359787393 +0000 UTC m=+0.477180970,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-94.ec2.internal,}" May 28 20:52:30.383882 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.383733 2563 manager.go:324] Recovery completed May 28 20:52:30.387908 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.387896 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:52:30.389561 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.389542 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientMemory" May 28 20:52:30.389611 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.389575 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:52:30.389611 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.389588 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientPID" May 28 20:52:30.390032 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.390020 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" May 28 20:52:30.390076 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.390031 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 28 20:52:30.390076 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.390053 2563 state_mem.go:36] "Initialized new in-memory state store" May 28 20:52:30.392122 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.392110 2563 policy_none.go:49] "None policy: Start" May 28 20:52:30.392210 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.392127 2563 memory_manager.go:186] "Starting memorymanager" policy="None" May 28 20:52:30.392210 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.392136 2563 state_mem.go:35] "Initializing new in-memory state store" May 28 20:52:30.392497 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.392439 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-94.ec2.internal.18b3d5faae9d4879 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-94.ec2.internal,UID:ip-10-0-140-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-94.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-94.ec2.internal,},FirstTimestamp:2026-05-28 20:52:30.389561465 +0000 UTC m=+0.506955041,LastTimestamp:2026-05-28 20:52:30.389561465 +0000 UTC m=+0.506955041,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-94.ec2.internal,}" May 28 20:52:30.402096 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.402079 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xxpgz" May 28 20:52:30.402869 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.402812 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-94.ec2.internal.18b3d5faae9d9cc4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-94.ec2.internal,UID:ip-10-0-140-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-94.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-94.ec2.internal,},FirstTimestamp:2026-05-28 20:52:30.389583044 +0000 UTC m=+0.506976619,LastTimestamp:2026-05-28 20:52:30.389583044 +0000 UTC m=+0.506976619,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-94.ec2.internal,}" May 28 20:52:30.413354 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.413297 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-94.ec2.internal.18b3d5faae9dc281 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-94.ec2.internal,UID:ip-10-0-140-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-94.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-94.ec2.internal,},FirstTimestamp:2026-05-28 20:52:30.389592705 +0000 UTC m=+0.506986282,LastTimestamp:2026-05-28 20:52:30.389592705 +0000 UTC m=+0.506986282,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-94.ec2.internal,}" May 28 20:52:30.415757 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.415742 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xxpgz" May 28 20:52:30.448743 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.438729 2563 manager.go:341] "Starting Device Plugin manager" May 28 20:52:30.448743 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.438753 2563 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 28 20:52:30.448743 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.438763 2563 server.go:85] "Starting device plugin registration server" May 28 20:52:30.448743 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.438976 2563 eviction_manager.go:189] "Eviction manager: starting control loop" May 28 20:52:30.448743 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.438989 2563 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 28 20:52:30.448743 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.439056 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 28 20:52:30.448743 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.439139 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 28 20:52:30.448743 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.439150 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 28 20:52:30.448743 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.439717 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 28 20:52:30.448743 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.439749 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:30.461278 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.461254 2563 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 28 20:52:30.462524 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.462507 2563 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 28 20:52:30.462591 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.462533 2563 status_manager.go:230] "Starting to sync pod status with apiserver" May 28 20:52:30.462591 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.462552 2563 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 28 20:52:30.462591 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.462561 2563 kubelet.go:2451] "Starting kubelet main sync loop" May 28 20:52:30.462685 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.462595 2563 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 28 20:52:30.467378 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.467361 2563 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 20:52:30.539955 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.539911 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:52:30.541191 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.541159 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientMemory" May 28 20:52:30.541264 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.541206 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:52:30.541264 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.541217 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientPID" May 28 20:52:30.541264 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.541250 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-94.ec2.internal" May 28 20:52:30.553181 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.553145 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-94.ec2.internal" May 28 20:52:30.553181 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.553182 2563 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-94.ec2.internal\": node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:30.563086 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.563068 2563 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-94.ec2.internal"] May 28 20:52:30.563198 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.563186 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:52:30.564861 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.564847 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientMemory" May 28 20:52:30.564930 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.564872 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:52:30.564930 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.564881 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientPID" May 28 20:52:30.565952 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.565941 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:52:30.566143 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.566127 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" May 28 20:52:30.566242 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.566184 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:52:30.566852 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.566836 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientMemory" May 28 20:52:30.566923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.566860 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:52:30.566923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.566870 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientPID" May 28 20:52:30.566923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.566873 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientMemory" May 28 20:52:30.566923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.566888 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:52:30.566923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.566897 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientPID" May 28 20:52:30.567888 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.567874 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-94.ec2.internal" May 28 20:52:30.567963 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.567897 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:52:30.568515 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.568492 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be9633c9382b980a365833a255350af4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-94.ec2.internal\" (UID: \"be9633c9382b980a365833a255350af4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-94.ec2.internal" May 28 20:52:30.568576 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.568524 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f2a4680f12c0c1508603c7999a91533-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal\" (UID: \"8f2a4680f12c0c1508603c7999a91533\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" May 28 20:52:30.568576 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.568551 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2a4680f12c0c1508603c7999a91533-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal\" (UID: \"8f2a4680f12c0c1508603c7999a91533\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" May 28 20:52:30.568825 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.568811 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientMemory" May 28 20:52:30.568885 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.568836 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:52:30.568885 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.568846 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeHasSufficientPID" May 28 20:52:30.584227 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.584211 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:30.599712 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.599687 2563 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-94.ec2.internal\" not found" node="ip-10-0-140-94.ec2.internal" May 28 20:52:30.604081 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.604067 2563 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-94.ec2.internal\" not found" node="ip-10-0-140-94.ec2.internal" May 28 20:52:30.669306 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.669275 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be9633c9382b980a365833a255350af4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-94.ec2.internal\" (UID: \"be9633c9382b980a365833a255350af4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-94.ec2.internal" May 28 20:52:30.669411 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.669313 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f2a4680f12c0c1508603c7999a91533-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal\" (UID: \"8f2a4680f12c0c1508603c7999a91533\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" May 28 20:52:30.669411 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.669338 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2a4680f12c0c1508603c7999a91533-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal\" (UID: \"8f2a4680f12c0c1508603c7999a91533\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" May 28 20:52:30.669411 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.669394 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2a4680f12c0c1508603c7999a91533-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal\" (UID: \"8f2a4680f12c0c1508603c7999a91533\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" May 28 20:52:30.669411 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.669398 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f2a4680f12c0c1508603c7999a91533-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal\" (UID: \"8f2a4680f12c0c1508603c7999a91533\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" May 28 20:52:30.669411 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.669398 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be9633c9382b980a365833a255350af4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-94.ec2.internal\" (UID: \"be9633c9382b980a365833a255350af4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-94.ec2.internal" May 28 20:52:30.684440 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.684421 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:30.784581 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.784556 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:30.885134 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.885066 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:30.902283 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.902266 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" May 28 20:52:30.906969 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:30.906953 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-94.ec2.internal" May 28 20:52:30.985885 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:30.985845 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:31.086659 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:31.086633 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:31.187332 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:31.187266 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:31.267607 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.267579 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 28 20:52:31.268057 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.267727 2563 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 28 20:52:31.287967 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:31.287946 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:31.367006 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.366987 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 28 20:52:31.388611 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:31.388595 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:31.402772 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.402749 2563 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 20:52:31.417389 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.417362 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 20:47:30 +0000 UTC" deadline="2027-11-28 16:42:22.62815999 +0000 UTC" May 28 20:52:31.417504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.417389 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13171h49m51.21077501s" May 28 20:52:31.443085 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.443036 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ndr26" May 28 20:52:31.453894 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.453869 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ndr26" May 28 20:52:31.477258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.477239 2563 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 28 20:52:31.489205 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:31.489184 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:31.557542 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:31.557512 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe9633c9382b980a365833a255350af4.slice/crio-bce2ddbd4650a1833d76855e6f1a2ad3ed21574926c8d736750380a216631d83 WatchSource:0}: Error finding container bce2ddbd4650a1833d76855e6f1a2ad3ed21574926c8d736750380a216631d83: Status 404 returned error can't find the container with id bce2ddbd4650a1833d76855e6f1a2ad3ed21574926c8d736750380a216631d83 May 28 20:52:31.557744 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:31.557710 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f2a4680f12c0c1508603c7999a91533.slice/crio-7486b0656d4a1ccb5bef4aceb04f8ae32e0620b797bc933478a31cb7949d8e2b WatchSource:0}: Error finding container 7486b0656d4a1ccb5bef4aceb04f8ae32e0620b797bc933478a31cb7949d8e2b: Status 404 returned error can't find the container with id 7486b0656d4a1ccb5bef4aceb04f8ae32e0620b797bc933478a31cb7949d8e2b May 28 20:52:31.560188 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.560157 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 20:52:31.589713 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:31.589694 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-94.ec2.internal\" not found" May 28 20:52:31.650752 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.650729 2563 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 28 20:52:31.668306 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.668285 2563 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" May 28 20:52:31.691118 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.691098 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 20:52:31.692350 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.692335 2563 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-94.ec2.internal" May 28 20:52:31.703081 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.703041 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 20:52:31.821718 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:31.821692 2563 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 28 20:52:32.347018 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.346982 2563 apiserver.go:52] "Watching apiserver" May 28 20:52:32.354368 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.354325 2563 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 28 20:52:32.354773 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.354734 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-l89rg","openshift-multus/network-metrics-daemon-qqths","openshift-network-diagnostics/network-check-target-45r74","kube-system/konnectivity-agent-2nxmw","openshift-dns/node-resolver-lkcmf","openshift-multus/multus-additional-cni-plugins-4tz8l","openshift-network-operator/iptables-alerter-qsrfx","openshift-ovn-kubernetes/ovnkube-node-nlv9g","kube-system/kube-apiserver-proxy-ip-10-0-140-94.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk","openshift-cluster-node-tuning-operator/tuned-kbnwf","openshift-image-registry/node-ca-9lkdc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal"] May 28 20:52:32.357452 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.357419 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qsrfx" May 28 20:52:32.359676 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.359643 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:32.359771 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:32.359715 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:32.363812 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.363789 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:32.363984 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.363966 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 28 20:52:32.364142 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.364098 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 28 20:52:32.364263 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.364227 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-blvfp\"" May 28 20:52:32.364333 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.364101 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 28 20:52:32.366268 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.366247 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lkcmf" May 28 20:52:32.368182 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.368152 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 28 20:52:32.368982 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.368601 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c94qv\"" May 28 20:52:32.368982 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.368602 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.369130 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.369082 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 28 20:52:32.369130 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.369111 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 28 20:52:32.369281 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.369261 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zcl57\"" May 28 20:52:32.369373 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.369273 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 28 20:52:32.371348 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.371326 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l89rg" May 28 20:52:32.373874 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.373854 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:32.373966 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:32.373930 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:32.374025 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.373988 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.376291 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.376272 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.377998 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.377976 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 28 20:52:32.378086 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.378056 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wf6w9\"" May 28 20:52:32.378951 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.378930 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.379052 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.378965 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/946679ad-8718-48c0-98c2-03feb3f67b30-hosts-file\") pod \"node-resolver-lkcmf\" (UID: \"946679ad-8718-48c0-98c2-03feb3f67b30\") " pod="openshift-dns/node-resolver-lkcmf" May 28 20:52:32.379052 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.378990 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.379052 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379016 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.379052 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379040 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b289084f-e171-435e-8cf2-d5ba9965c060-konnectivity-ca\") pod \"konnectivity-agent-2nxmw\" (UID: \"b289084f-e171-435e-8cf2-d5ba9965c060\") " pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:32.379261 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379063 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkzgp\" (UniqueName: \"kubernetes.io/projected/946679ad-8718-48c0-98c2-03feb3f67b30-kube-api-access-gkzgp\") pod \"node-resolver-lkcmf\" (UID: \"946679ad-8718-48c0-98c2-03feb3f67b30\") " pod="openshift-dns/node-resolver-lkcmf" May 28 20:52:32.379261 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379087 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-os-release\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.379261 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379111 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5-iptables-alerter-script\") pod \"iptables-alerter-qsrfx\" (UID: \"0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5\") " pod="openshift-network-operator/iptables-alerter-qsrfx" May 28 20:52:32.379261 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379152 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5-host-slash\") pod \"iptables-alerter-qsrfx\" (UID: \"0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5\") " pod="openshift-network-operator/iptables-alerter-qsrfx" May 28 20:52:32.379261 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379196 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:32.379261 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379217 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gfs4\" (UniqueName: \"kubernetes.io/projected/338d9053-7caa-48f3-bd43-6433cf054fc5-kube-api-access-6gfs4\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:32.379261 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379232 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.379589 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379277 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw6vb\" (UniqueName: \"kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb\") pod \"network-check-target-45r74\" (UID: \"5c11e3de-f375-4994-9fb5-26a89d3e0ec7\") " pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:32.379589 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379301 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b289084f-e171-435e-8cf2-d5ba9965c060-agent-certs\") pod \"konnectivity-agent-2nxmw\" (UID: \"b289084f-e171-435e-8cf2-d5ba9965c060\") " pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:32.379589 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379322 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-system-cni-dir\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.379589 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379342 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-cnibin\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.379589 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946679ad-8718-48c0-98c2-03feb3f67b30-tmp-dir\") pod \"node-resolver-lkcmf\" (UID: \"946679ad-8718-48c0-98c2-03feb3f67b30\") " pod="openshift-dns/node-resolver-lkcmf" May 28 20:52:32.379589 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379386 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mcpt\" (UniqueName: \"kubernetes.io/projected/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-kube-api-access-8mcpt\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.379589 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.379403 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trjtw\" (UniqueName: \"kubernetes.io/projected/0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5-kube-api-access-trjtw\") pod \"iptables-alerter-qsrfx\" (UID: \"0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5\") " pod="openshift-network-operator/iptables-alerter-qsrfx" May 28 20:52:32.380826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.380799 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9lkdc" May 28 20:52:32.380901 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.380832 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.381377 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.381360 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 28 20:52:32.381559 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.381540 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 28 20:52:32.381687 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.381605 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 28 20:52:32.381831 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.381810 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 28 20:52:32.381899 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.381854 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 28 20:52:32.381899 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.381861 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 28 20:52:32.381899 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.381856 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 28 20:52:32.382036 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.381888 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 28 20:52:32.382036 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.381980 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 28 20:52:32.382207 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.382184 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 28 20:52:32.382416 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.382398 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-289j5\"" May 28 20:52:32.382511 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.382422 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4qgsc\"" May 28 20:52:32.383337 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.383309 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 28 20:52:32.386054 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.386036 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 28 20:52:32.386054 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.386049 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 28 20:52:32.386346 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.386330 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 28 20:52:32.387685 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.387665 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 28 20:52:32.387977 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.387961 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xnnl8\"" May 28 20:52:32.388236 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.388218 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 28 20:52:32.388565 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.388433 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 28 20:52:32.388565 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.388434 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 28 20:52:32.388565 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.388471 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ffz75\"" May 28 20:52:32.388565 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.388529 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 28 20:52:32.388565 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.388555 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mwhzj\"" May 28 20:52:32.454996 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.454970 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 20:47:31 +0000 UTC" deadline="2027-11-28 09:36:59.819946282 +0000 UTC" May 28 20:52:32.454996 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.454995 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13164h44m27.364954355s" May 28 20:52:32.468038 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.467997 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-94.ec2.internal" event={"ID":"be9633c9382b980a365833a255350af4","Type":"ContainerStarted","Data":"bce2ddbd4650a1833d76855e6f1a2ad3ed21574926c8d736750380a216631d83"} May 28 20:52:32.468655 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.468637 2563 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 28 20:52:32.469002 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.468981 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" event={"ID":"8f2a4680f12c0c1508603c7999a91533","Type":"ContainerStarted","Data":"7486b0656d4a1ccb5bef4aceb04f8ae32e0620b797bc933478a31cb7949d8e2b"} May 28 20:52:32.479852 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.479831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-socket-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.479955 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.479873 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-kubernetes\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.479955 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.479901 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-var-lib-openvswitch\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.479955 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.479928 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-system-cni-dir\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.480111 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.479975 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-os-release\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.480111 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.479999 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-conf-dir\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.480111 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480024 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-run-multus-certs\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.480111 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480066 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-sysconfig\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.480309 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480129 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-sys\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.480309 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480149 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.480309 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480189 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.480309 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480225 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-var-lib-cni-multus\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.480309 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480298 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-kubelet-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.480541 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480328 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hwr2\" (UniqueName: \"kubernetes.io/projected/c225cfeb-ded5-4744-989c-3cc9f0eb2136-kube-api-access-2hwr2\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.480541 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480389 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-run-openvswitch\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.480541 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480413 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-cni-bin\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.480541 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.480541 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480479 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:32.480541 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480507 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-var-lib-kubelet\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.480775 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480560 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-etc-selinux\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.480906 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480887 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.481012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.480987 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-run-netns\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.481112 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.481026 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-daemon-config\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.481112 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.481104 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-ovnkube-config\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.481233 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.481125 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-cnibin\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.481233 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.481163 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/943ced3a-27aa-4fb2-8ffd-53d1269faf80-serviceca\") pod \"node-ca-9lkdc\" (UID: \"943ced3a-27aa-4fb2-8ffd-53d1269faf80\") " pod="openshift-image-registry/node-ca-9lkdc" May 28 20:52:32.481233 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.481211 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-kubelet\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.481373 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.481280 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-etc-openvswitch\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.481373 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.481345 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-run-k8s-cni-cncf-io\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.481472 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:32.481444 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:32.481550 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.481528 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-cni-binary-copy\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.481604 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.481546 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-tuned\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.481711 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.481684 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.481804 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:32.481792 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs podName:338d9053-7caa-48f3-bd43-6433cf054fc5 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:32.981760631 +0000 UTC m=+3.099154210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs") pod "network-metrics-daemon-qqths" (UID: "338d9053-7caa-48f3-bd43-6433cf054fc5") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:32.481985 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.481967 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-tmp\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.482092 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482005 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-run-systemd\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.482092 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482052 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-run-ovn\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.482687 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482143 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-env-overrides\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.482773 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482722 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ac6fddb-e201-4648-8670-9a00e75dd19f-cni-binary-copy\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.482773 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482756 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-socket-dir-parent\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.482869 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482794 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b289084f-e171-435e-8cf2-d5ba9965c060-konnectivity-ca\") pod \"konnectivity-agent-2nxmw\" (UID: \"b289084f-e171-435e-8cf2-d5ba9965c060\") " pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:32.482869 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482822 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6gfs4\" (UniqueName: \"kubernetes.io/projected/338d9053-7caa-48f3-bd43-6433cf054fc5-kube-api-access-6gfs4\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:32.482957 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482868 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/943ced3a-27aa-4fb2-8ffd-53d1269faf80-host\") pod \"node-ca-9lkdc\" (UID: \"943ced3a-27aa-4fb2-8ffd-53d1269faf80\") " pod="openshift-image-registry/node-ca-9lkdc" May 28 20:52:32.482957 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482898 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxtnd\" (UniqueName: \"kubernetes.io/projected/943ced3a-27aa-4fb2-8ffd-53d1269faf80-kube-api-access-jxtnd\") pod \"node-ca-9lkdc\" (UID: \"943ced3a-27aa-4fb2-8ffd-53d1269faf80\") " pod="openshift-image-registry/node-ca-9lkdc" May 28 20:52:32.482957 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482927 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-systemd-units\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.483089 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482955 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-var-lib-kubelet\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.483089 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.482982 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8mcpt\" (UniqueName: \"kubernetes.io/projected/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-kube-api-access-8mcpt\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.483089 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483012 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-cnibin\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.483089 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483043 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-modprobe-d\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.483089 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483073 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-host\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.483315 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483101 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-cni-netd\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.483315 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483127 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/946679ad-8718-48c0-98c2-03feb3f67b30-hosts-file\") pod \"node-resolver-lkcmf\" (UID: \"946679ad-8718-48c0-98c2-03feb3f67b30\") " pod="openshift-dns/node-resolver-lkcmf" May 28 20:52:32.483315 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-sysctl-d\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.483315 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483204 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-lib-modules\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.483315 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483237 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsj57\" (UniqueName: \"kubernetes.io/projected/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-kube-api-access-zsj57\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.483315 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483270 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5-iptables-alerter-script\") pod \"iptables-alerter-qsrfx\" (UID: \"0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5\") " pod="openshift-network-operator/iptables-alerter-qsrfx" May 28 20:52:32.483315 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483305 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-cni-dir\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.483608 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483332 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-var-lib-cni-bin\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.483608 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483362 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kfvw\" (UniqueName: \"kubernetes.io/projected/9ac6fddb-e201-4648-8670-9a00e75dd19f-kube-api-access-4kfvw\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.483608 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483389 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-sys-fs\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.483608 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-sysctl-conf\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.483608 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483443 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-run\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.483608 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483467 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-run-netns\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.483608 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483501 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gw6vb\" (UniqueName: \"kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb\") pod \"network-check-target-45r74\" (UID: \"5c11e3de-f375-4994-9fb5-26a89d3e0ec7\") " pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:32.483608 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483531 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b289084f-e171-435e-8cf2-d5ba9965c060-agent-certs\") pod \"konnectivity-agent-2nxmw\" (UID: \"b289084f-e171-435e-8cf2-d5ba9965c060\") " pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:32.483608 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483562 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-registration-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.483608 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-device-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483618 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-slash\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483648 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-log-socket\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483676 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-ovnkube-script-lib\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483708 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946679ad-8718-48c0-98c2-03feb3f67b30-tmp-dir\") pod \"node-resolver-lkcmf\" (UID: \"946679ad-8718-48c0-98c2-03feb3f67b30\") " pod="openshift-dns/node-resolver-lkcmf" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483741 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-hostroot\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483774 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483807 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483834 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gkzgp\" (UniqueName: \"kubernetes.io/projected/946679ad-8718-48c0-98c2-03feb3f67b30-kube-api-access-gkzgp\") pod \"node-resolver-lkcmf\" (UID: \"946679ad-8718-48c0-98c2-03feb3f67b30\") " pod="openshift-dns/node-resolver-lkcmf" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483866 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-os-release\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483894 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5-host-slash\") pod \"iptables-alerter-qsrfx\" (UID: \"0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5\") " pod="openshift-network-operator/iptables-alerter-qsrfx" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483924 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-etc-kubernetes\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483954 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-node-log\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.484012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.483985 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-ovn-node-metrics-cert\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.484598 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.484019 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9zlh\" (UniqueName: \"kubernetes.io/projected/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-kube-api-access-z9zlh\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.484598 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.484047 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-systemd\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.484598 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.484079 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-trjtw\" (UniqueName: \"kubernetes.io/projected/0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5-kube-api-access-trjtw\") pod \"iptables-alerter-qsrfx\" (UID: \"0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5\") " pod="openshift-network-operator/iptables-alerter-qsrfx" May 28 20:52:32.484598 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.484113 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-system-cni-dir\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.484789 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.484703 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 28 20:52:32.484789 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.484761 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5-iptables-alerter-script\") pod \"iptables-alerter-qsrfx\" (UID: \"0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5\") " pod="openshift-network-operator/iptables-alerter-qsrfx" May 28 20:52:32.484919 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.484855 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.484919 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.484889 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/946679ad-8718-48c0-98c2-03feb3f67b30-hosts-file\") pod \"node-resolver-lkcmf\" (UID: \"946679ad-8718-48c0-98c2-03feb3f67b30\") " pod="openshift-dns/node-resolver-lkcmf" May 28 20:52:32.485013 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.484952 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946679ad-8718-48c0-98c2-03feb3f67b30-tmp-dir\") pod \"node-resolver-lkcmf\" (UID: \"946679ad-8718-48c0-98c2-03feb3f67b30\") " pod="openshift-dns/node-resolver-lkcmf" May 28 20:52:32.485083 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.485059 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-system-cni-dir\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.485140 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.485094 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5-host-slash\") pod \"iptables-alerter-qsrfx\" (UID: \"0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5\") " pod="openshift-network-operator/iptables-alerter-qsrfx" May 28 20:52:32.485211 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.485138 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-cnibin\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.485211 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.485160 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-os-release\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.485510 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.485464 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.485885 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.485663 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b289084f-e171-435e-8cf2-d5ba9965c060-konnectivity-ca\") pod \"konnectivity-agent-2nxmw\" (UID: \"b289084f-e171-435e-8cf2-d5ba9965c060\") " pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:32.488466 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.488446 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b289084f-e171-435e-8cf2-d5ba9965c060-agent-certs\") pod \"konnectivity-agent-2nxmw\" (UID: \"b289084f-e171-435e-8cf2-d5ba9965c060\") " pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:32.502665 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.502619 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkzgp\" (UniqueName: \"kubernetes.io/projected/946679ad-8718-48c0-98c2-03feb3f67b30-kube-api-access-gkzgp\") pod \"node-resolver-lkcmf\" (UID: \"946679ad-8718-48c0-98c2-03feb3f67b30\") " pod="openshift-dns/node-resolver-lkcmf" May 28 20:52:32.503928 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.503182 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trjtw\" (UniqueName: \"kubernetes.io/projected/0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5-kube-api-access-trjtw\") pod \"iptables-alerter-qsrfx\" (UID: \"0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5\") " pod="openshift-network-operator/iptables-alerter-qsrfx" May 28 20:52:32.503928 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:32.503219 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:52:32.503928 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:32.503239 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:52:32.503928 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:32.503251 2563 projected.go:194] Error preparing data for projected volume kube-api-access-gw6vb for pod openshift-network-diagnostics/network-check-target-45r74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:32.503928 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:32.503307 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb podName:5c11e3de-f375-4994-9fb5-26a89d3e0ec7 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:33.00328927 +0000 UTC m=+3.120682854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gw6vb" (UniqueName: "kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb") pod "network-check-target-45r74" (UID: "5c11e3de-f375-4994-9fb5-26a89d3e0ec7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:32.503928 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.503679 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gfs4\" (UniqueName: \"kubernetes.io/projected/338d9053-7caa-48f3-bd43-6433cf054fc5-kube-api-access-6gfs4\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:32.505627 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.505582 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mcpt\" (UniqueName: \"kubernetes.io/projected/aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0-kube-api-access-8mcpt\") pod \"multus-additional-cni-plugins-4tz8l\" (UID: \"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0\") " pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.585438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585397 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-etc-selinux\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.585438 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585442 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-run-netns\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.585656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585466 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-daemon-config\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.585656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585482 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-ovnkube-config\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.585656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585513 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-cnibin\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.585656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585528 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-run-netns\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.585656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585538 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/943ced3a-27aa-4fb2-8ffd-53d1269faf80-serviceca\") pod \"node-ca-9lkdc\" (UID: \"943ced3a-27aa-4fb2-8ffd-53d1269faf80\") " pod="openshift-image-registry/node-ca-9lkdc" May 28 20:52:32.585656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585592 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-kubelet\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.585656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585593 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-etc-selinux\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.585656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585622 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-etc-openvswitch\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.585656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585650 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-run-k8s-cni-cncf-io\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.585656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585653 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-cnibin\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585654 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-kubelet\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585676 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-tuned\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585690 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-etc-openvswitch\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585702 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-tmp\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585728 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-run-systemd\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585771 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-run-systemd\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585702 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-run-k8s-cni-cncf-io\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585800 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-run-ovn\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585849 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-env-overrides\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585876 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ac6fddb-e201-4648-8670-9a00e75dd19f-cni-binary-copy\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585887 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-run-ovn\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585900 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-socket-dir-parent\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585925 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/943ced3a-27aa-4fb2-8ffd-53d1269faf80-host\") pod \"node-ca-9lkdc\" (UID: \"943ced3a-27aa-4fb2-8ffd-53d1269faf80\") " pod="openshift-image-registry/node-ca-9lkdc" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585975 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jxtnd\" (UniqueName: \"kubernetes.io/projected/943ced3a-27aa-4fb2-8ffd-53d1269faf80-kube-api-access-jxtnd\") pod \"node-ca-9lkdc\" (UID: \"943ced3a-27aa-4fb2-8ffd-53d1269faf80\") " pod="openshift-image-registry/node-ca-9lkdc" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.585992 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/943ced3a-27aa-4fb2-8ffd-53d1269faf80-serviceca\") pod \"node-ca-9lkdc\" (UID: \"943ced3a-27aa-4fb2-8ffd-53d1269faf80\") " pod="openshift-image-registry/node-ca-9lkdc" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586000 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-systemd-units\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586022 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-var-lib-kubelet\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.586097 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586048 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-modprobe-d\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586057 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/943ced3a-27aa-4fb2-8ffd-53d1269faf80-host\") pod \"node-ca-9lkdc\" (UID: \"943ced3a-27aa-4fb2-8ffd-53d1269faf80\") " pod="openshift-image-registry/node-ca-9lkdc" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586089 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-host\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586092 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-daemon-config\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586116 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-cni-netd\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586115 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-ovnkube-config\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586155 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-sysctl-d\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586158 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-cni-netd\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586190 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-modprobe-d\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586202 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-lib-modules\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586218 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-systemd-units\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586236 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zsj57\" (UniqueName: \"kubernetes.io/projected/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-kube-api-access-zsj57\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586264 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-cni-dir\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586287 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-env-overrides\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586291 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-var-lib-cni-bin\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586315 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-lib-modules\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586335 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4kfvw\" (UniqueName: \"kubernetes.io/projected/9ac6fddb-e201-4648-8670-9a00e75dd19f-kube-api-access-4kfvw\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586361 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-var-lib-kubelet\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.586993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586365 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-sys-fs\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586360 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-cni-dir\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586391 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-sysctl-conf\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586400 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-var-lib-cni-bin\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586414 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-run\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586412 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-socket-dir-parent\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586415 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-host\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586452 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-run-netns\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586493 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-sys-fs\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586511 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-run-netns\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586516 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-sysctl-d\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586544 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-registration-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586572 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-run\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586572 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-device-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586617 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-slash\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586649 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ac6fddb-e201-4648-8670-9a00e75dd19f-cni-binary-copy\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586663 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-log-socket\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586653 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-sysctl-conf\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.587829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586693 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-ovnkube-script-lib\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586696 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-slash\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586704 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-registration-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586731 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-device-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586725 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-log-socket\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586774 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-hostroot\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586832 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-hostroot\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586846 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-etc-kubernetes\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586875 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-node-log\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586899 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-ovn-node-metrics-cert\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586924 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z9zlh\" (UniqueName: \"kubernetes.io/projected/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-kube-api-access-z9zlh\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586938 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-etc-kubernetes\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586947 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-systemd\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586967 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-node-log\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.586976 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-socket-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587025 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-systemd\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587129 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-kubernetes\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587157 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-socket-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.588632 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587185 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-var-lib-openvswitch\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587215 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-system-cni-dir\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587232 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-kubernetes\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587243 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-os-release\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587245 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-var-lib-openvswitch\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587267 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-conf-dir\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587269 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-ovnkube-script-lib\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587280 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-system-cni-dir\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587312 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-run-multus-certs\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587321 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-os-release\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587343 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-sysconfig\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587357 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-multus-conf-dir\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587370 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-sys\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587387 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-run-multus-certs\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587395 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587405 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-sysconfig\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587422 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-var-lib-cni-multus\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587441 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-sys\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.589258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587447 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-kubelet-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587450 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587473 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2hwr2\" (UniqueName: \"kubernetes.io/projected/c225cfeb-ded5-4744-989c-3cc9f0eb2136-kube-api-access-2hwr2\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587517 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-run-openvswitch\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587536 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c225cfeb-ded5-4744-989c-3cc9f0eb2136-kubelet-dir\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587545 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-cni-bin\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587577 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-var-lib-cni-multus\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587577 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587605 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-run-openvswitch\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587615 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-cni-bin\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587626 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-var-lib-kubelet\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587652 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.587656 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ac6fddb-e201-4648-8670-9a00e75dd19f-host-var-lib-kubelet\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.588295 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-tmp\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.588526 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-etc-tuned\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.589826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.589120 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-ovn-node-metrics-cert\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.605459 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.605406 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kfvw\" (UniqueName: \"kubernetes.io/projected/9ac6fddb-e201-4648-8670-9a00e75dd19f-kube-api-access-4kfvw\") pod \"multus-l89rg\" (UID: \"9ac6fddb-e201-4648-8670-9a00e75dd19f\") " pod="openshift-multus/multus-l89rg" May 28 20:52:32.605554 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.605494 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxtnd\" (UniqueName: \"kubernetes.io/projected/943ced3a-27aa-4fb2-8ffd-53d1269faf80-kube-api-access-jxtnd\") pod \"node-ca-9lkdc\" (UID: \"943ced3a-27aa-4fb2-8ffd-53d1269faf80\") " pod="openshift-image-registry/node-ca-9lkdc" May 28 20:52:32.611878 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.611854 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9zlh\" (UniqueName: \"kubernetes.io/projected/e6f5ed42-ee6c-4a50-a715-93c3b4e92da8-kube-api-access-z9zlh\") pod \"ovnkube-node-nlv9g\" (UID: \"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.612098 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.612057 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hwr2\" (UniqueName: \"kubernetes.io/projected/c225cfeb-ded5-4744-989c-3cc9f0eb2136-kube-api-access-2hwr2\") pod \"aws-ebs-csi-driver-node-59kqk\" (UID: \"c225cfeb-ded5-4744-989c-3cc9f0eb2136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.614954 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.614935 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsj57\" (UniqueName: \"kubernetes.io/projected/40f60f87-8c1d-4092-96b5-d7ab2bcb1c68-kube-api-access-zsj57\") pod \"tuned-kbnwf\" (UID: \"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68\") " pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.669540 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.669508 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qsrfx" May 28 20:52:32.681191 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.681155 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:32.693236 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.693209 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lkcmf" May 28 20:52:32.696835 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.696816 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4tz8l" May 28 20:52:32.705406 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.705378 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l89rg" May 28 20:52:32.713366 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.713349 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:32.720754 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.720735 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" May 28 20:52:32.728241 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.728223 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9lkdc" May 28 20:52:32.733812 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.733795 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" May 28 20:52:32.832095 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.832068 2563 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 20:52:32.990163 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:32.990081 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:32.990319 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:32.990262 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:32.990372 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:32.990340 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs podName:338d9053-7caa-48f3-bd43-6433cf054fc5 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:33.990322265 +0000 UTC m=+4.107715831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs") pod "network-metrics-daemon-qqths" (UID: "338d9053-7caa-48f3-bd43-6433cf054fc5") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:33.090751 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.090713 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gw6vb\" (UniqueName: \"kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb\") pod \"network-check-target-45r74\" (UID: \"5c11e3de-f375-4994-9fb5-26a89d3e0ec7\") " pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:33.090922 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:33.090834 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:52:33.090922 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:33.090854 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:52:33.090922 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:33.090865 2563 projected.go:194] Error preparing data for projected volume kube-api-access-gw6vb for pod openshift-network-diagnostics/network-check-target-45r74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:33.091048 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:33.090932 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb podName:5c11e3de-f375-4994-9fb5-26a89d3e0ec7 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:34.090912124 +0000 UTC m=+4.208305710 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw6vb" (UniqueName: "kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb") pod "network-check-target-45r74" (UID: "5c11e3de-f375-4994-9fb5-26a89d3e0ec7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:33.318634 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:33.318589 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc225cfeb_ded5_4744_989c_3cc9f0eb2136.slice/crio-52a98f948dbe8c79898e2e15ec71a6bea6c5296f06c3add6d4a71b8921492df9 WatchSource:0}: Error finding container 52a98f948dbe8c79898e2e15ec71a6bea6c5296f06c3add6d4a71b8921492df9: Status 404 returned error can't find the container with id 52a98f948dbe8c79898e2e15ec71a6bea6c5296f06c3add6d4a71b8921492df9 May 28 20:52:33.319717 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:33.319695 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa3c1cc6_50e9_45e4_aa35_5028cc0c85a0.slice/crio-c13a264570ecb84330b5b64a08cb2d3fed4e817896ab23459374884ca3011c92 WatchSource:0}: Error finding container c13a264570ecb84330b5b64a08cb2d3fed4e817896ab23459374884ca3011c92: Status 404 returned error can't find the container with id c13a264570ecb84330b5b64a08cb2d3fed4e817896ab23459374884ca3011c92 May 28 20:52:33.320337 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:33.320185 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943ced3a_27aa_4fb2_8ffd_53d1269faf80.slice/crio-bd62e1082e7558dc33d382f41e2771237e256d3f3be8907926d2c147263ce080 WatchSource:0}: Error finding container bd62e1082e7558dc33d382f41e2771237e256d3f3be8907926d2c147263ce080: Status 404 returned error can't find the container with id bd62e1082e7558dc33d382f41e2771237e256d3f3be8907926d2c147263ce080 May 28 20:52:33.323202 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:33.323161 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40f60f87_8c1d_4092_96b5_d7ab2bcb1c68.slice/crio-7a6de35f8e1914867660a39083450cbe6aadb3a55a4a3204dbccb66e01644e2a WatchSource:0}: Error finding container 7a6de35f8e1914867660a39083450cbe6aadb3a55a4a3204dbccb66e01644e2a: Status 404 returned error can't find the container with id 7a6de35f8e1914867660a39083450cbe6aadb3a55a4a3204dbccb66e01644e2a May 28 20:52:33.324565 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:33.324469 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946679ad_8718_48c0_98c2_03feb3f67b30.slice/crio-348e31ceba22a2b5a0a4f0d5c349a167eb0cd5f3fbcb8ea95fe10d6a001266d8 WatchSource:0}: Error finding container 348e31ceba22a2b5a0a4f0d5c349a167eb0cd5f3fbcb8ea95fe10d6a001266d8: Status 404 returned error can't find the container with id 348e31ceba22a2b5a0a4f0d5c349a167eb0cd5f3fbcb8ea95fe10d6a001266d8 May 28 20:52:33.325375 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:33.325351 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f5ed42_ee6c_4a50_a715_93c3b4e92da8.slice/crio-8e33cdb9873b6f158e7b8a0885a873bbc8f3fa32839973a6a78db68da78e9bac WatchSource:0}: Error finding container 8e33cdb9873b6f158e7b8a0885a873bbc8f3fa32839973a6a78db68da78e9bac: Status 404 returned error can't find the container with id 8e33cdb9873b6f158e7b8a0885a873bbc8f3fa32839973a6a78db68da78e9bac May 28 20:52:33.330750 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:33.330563 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb289084f_e171_435e_8cf2_d5ba9965c060.slice/crio-83489d2955c9fa1e756cf4aae22255afa87e00ac35e8866b30c7e6d097ccc300 WatchSource:0}: Error finding container 83489d2955c9fa1e756cf4aae22255afa87e00ac35e8866b30c7e6d097ccc300: Status 404 returned error can't find the container with id 83489d2955c9fa1e756cf4aae22255afa87e00ac35e8866b30c7e6d097ccc300 May 28 20:52:33.331876 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:52:33.331854 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac6fddb_e201_4648_8670_9a00e75dd19f.slice/crio-3ad8f06fa675742069cd2b91704d09ab828010d096eb6ad75d6deabb5c5584cb WatchSource:0}: Error finding container 3ad8f06fa675742069cd2b91704d09ab828010d096eb6ad75d6deabb5c5584cb: Status 404 returned error can't find the container with id 3ad8f06fa675742069cd2b91704d09ab828010d096eb6ad75d6deabb5c5584cb May 28 20:52:33.455776 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.455744 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 20:47:31 +0000 UTC" deadline="2028-02-24 11:46:24.811876746 +0000 UTC" May 28 20:52:33.455776 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.455772 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15278h53m51.356106837s" May 28 20:52:33.462988 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.462969 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:33.463082 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:33.463065 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:33.473281 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.473245 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" event={"ID":"c225cfeb-ded5-4744-989c-3cc9f0eb2136","Type":"ContainerStarted","Data":"52a98f948dbe8c79898e2e15ec71a6bea6c5296f06c3add6d4a71b8921492df9"} May 28 20:52:33.476152 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.476126 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-94.ec2.internal" event={"ID":"be9633c9382b980a365833a255350af4","Type":"ContainerStarted","Data":"d6ac80c34e328ee652ce9819736578370ebda5c8eeffa4d3efc49c0a5bae0030"} May 28 20:52:33.479481 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.479455 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2nxmw" event={"ID":"b289084f-e171-435e-8cf2-d5ba9965c060","Type":"ContainerStarted","Data":"83489d2955c9fa1e756cf4aae22255afa87e00ac35e8866b30c7e6d097ccc300"} May 28 20:52:33.480502 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.480473 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" event={"ID":"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8","Type":"ContainerStarted","Data":"8e33cdb9873b6f158e7b8a0885a873bbc8f3fa32839973a6a78db68da78e9bac"} May 28 20:52:33.481375 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.481355 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" event={"ID":"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68","Type":"ContainerStarted","Data":"7a6de35f8e1914867660a39083450cbe6aadb3a55a4a3204dbccb66e01644e2a"} May 28 20:52:33.485665 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.485642 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l89rg" event={"ID":"9ac6fddb-e201-4648-8670-9a00e75dd19f","Type":"ContainerStarted","Data":"3ad8f06fa675742069cd2b91704d09ab828010d096eb6ad75d6deabb5c5584cb"} May 28 20:52:33.489000 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.488974 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lkcmf" event={"ID":"946679ad-8718-48c0-98c2-03feb3f67b30","Type":"ContainerStarted","Data":"348e31ceba22a2b5a0a4f0d5c349a167eb0cd5f3fbcb8ea95fe10d6a001266d8"} May 28 20:52:33.490456 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.490434 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qsrfx" event={"ID":"0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5","Type":"ContainerStarted","Data":"c2990f32aaf23766b93397fef4a8d9e7aaa7e9eb2fffc9055abfde31349c4318"} May 28 20:52:33.491404 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.491383 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9lkdc" event={"ID":"943ced3a-27aa-4fb2-8ffd-53d1269faf80","Type":"ContainerStarted","Data":"bd62e1082e7558dc33d382f41e2771237e256d3f3be8907926d2c147263ce080"} May 28 20:52:33.492376 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.492354 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4tz8l" event={"ID":"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0","Type":"ContainerStarted","Data":"c13a264570ecb84330b5b64a08cb2d3fed4e817896ab23459374884ca3011c92"} May 28 20:52:33.998077 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:33.997991 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:33.998280 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:33.998154 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:33.998280 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:33.998244 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs podName:338d9053-7caa-48f3-bd43-6433cf054fc5 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:35.998222396 +0000 UTC m=+6.115615959 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs") pod "network-metrics-daemon-qqths" (UID: "338d9053-7caa-48f3-bd43-6433cf054fc5") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:34.098682 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:34.098645 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gw6vb\" (UniqueName: \"kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb\") pod \"network-check-target-45r74\" (UID: \"5c11e3de-f375-4994-9fb5-26a89d3e0ec7\") " pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:34.098946 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:34.098829 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:52:34.098946 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:34.098848 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:52:34.098946 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:34.098862 2563 projected.go:194] Error preparing data for projected volume kube-api-access-gw6vb for pod openshift-network-diagnostics/network-check-target-45r74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:34.098946 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:34.098921 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb podName:5c11e3de-f375-4994-9fb5-26a89d3e0ec7 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:36.098902665 +0000 UTC m=+6.216296229 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw6vb" (UniqueName: "kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb") pod "network-check-target-45r74" (UID: "5c11e3de-f375-4994-9fb5-26a89d3e0ec7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:34.465841 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:34.465310 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:34.465841 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:34.465418 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:34.507377 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:34.506680 2563 generic.go:358] "Generic (PLEG): container finished" podID="8f2a4680f12c0c1508603c7999a91533" containerID="717b88cc13fe94578bfaccad9bce916d4d73333d6a4fe8c80c499dc92d0d9583" exitCode=0 May 28 20:52:34.507377 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:34.506787 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" event={"ID":"8f2a4680f12c0c1508603c7999a91533","Type":"ContainerDied","Data":"717b88cc13fe94578bfaccad9bce916d4d73333d6a4fe8c80c499dc92d0d9583"} May 28 20:52:34.527271 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:34.527119 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-94.ec2.internal" podStartSLOduration=3.527102394 podStartE2EDuration="3.527102394s" podCreationTimestamp="2026-05-28 20:52:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 20:52:33.506680596 +0000 UTC m=+3.624074180" watchObservedRunningTime="2026-05-28 20:52:34.527102394 +0000 UTC m=+4.644495981" May 28 20:52:35.148271 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.147501 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-xsvnt"] May 28 20:52:35.150819 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.150386 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:35.150819 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:35.150464 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:35.208450 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.208238 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15cde4c4-5e59-47e6-87f4-32de185a998e-kubelet-config\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:35.208450 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.208292 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15cde4c4-5e59-47e6-87f4-32de185a998e-dbus\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:35.208450 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.208359 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:35.309509 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.309474 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15cde4c4-5e59-47e6-87f4-32de185a998e-kubelet-config\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:35.309671 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.309527 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15cde4c4-5e59-47e6-87f4-32de185a998e-dbus\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:35.309671 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.309584 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:35.309782 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:35.309695 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 20:52:35.309782 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:35.309753 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret podName:15cde4c4-5e59-47e6-87f4-32de185a998e nodeName:}" failed. No retries permitted until 2026-05-28 20:52:35.809734644 +0000 UTC m=+5.927128226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret") pod "global-pull-secret-syncer-xsvnt" (UID: "15cde4c4-5e59-47e6-87f4-32de185a998e") : object "kube-system"/"original-pull-secret" not registered May 28 20:52:35.310067 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.310042 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15cde4c4-5e59-47e6-87f4-32de185a998e-kubelet-config\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:35.310187 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.310115 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15cde4c4-5e59-47e6-87f4-32de185a998e-dbus\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:35.463613 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.463532 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:35.463781 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:35.463748 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:35.519651 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.519607 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" event={"ID":"8f2a4680f12c0c1508603c7999a91533","Type":"ContainerStarted","Data":"d03ec0f0598c54dec3b00d2b6fce407414e1f93bf2c6a55a0669b4b05f62df95"} May 28 20:52:35.814545 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:35.814444 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:35.814709 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:35.814653 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 20:52:35.814770 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:35.814717 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret podName:15cde4c4-5e59-47e6-87f4-32de185a998e nodeName:}" failed. No retries permitted until 2026-05-28 20:52:36.814699591 +0000 UTC m=+6.932093157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret") pod "global-pull-secret-syncer-xsvnt" (UID: "15cde4c4-5e59-47e6-87f4-32de185a998e") : object "kube-system"/"original-pull-secret" not registered May 28 20:52:36.016964 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:36.016501 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:36.016964 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:36.016717 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:36.016964 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:36.016782 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs podName:338d9053-7caa-48f3-bd43-6433cf054fc5 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:40.016763369 +0000 UTC m=+10.134156944 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs") pod "network-metrics-daemon-qqths" (UID: "338d9053-7caa-48f3-bd43-6433cf054fc5") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:36.118406 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:36.117633 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gw6vb\" (UniqueName: \"kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb\") pod \"network-check-target-45r74\" (UID: \"5c11e3de-f375-4994-9fb5-26a89d3e0ec7\") " pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:36.118406 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:36.117843 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:52:36.118406 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:36.117863 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:52:36.118406 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:36.117877 2563 projected.go:194] Error preparing data for projected volume kube-api-access-gw6vb for pod openshift-network-diagnostics/network-check-target-45r74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:36.118406 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:36.117933 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb podName:5c11e3de-f375-4994-9fb5-26a89d3e0ec7 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:40.117916195 +0000 UTC m=+10.235309770 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw6vb" (UniqueName: "kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb") pod "network-check-target-45r74" (UID: "5c11e3de-f375-4994-9fb5-26a89d3e0ec7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:36.463633 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:36.463556 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:36.463633 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:36.463583 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:36.463839 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:36.463726 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:36.464225 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:36.464159 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:36.824443 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:36.824317 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:36.824852 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:36.824513 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 20:52:36.824852 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:36.824585 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret podName:15cde4c4-5e59-47e6-87f4-32de185a998e nodeName:}" failed. No retries permitted until 2026-05-28 20:52:38.824565903 +0000 UTC m=+8.941959472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret") pod "global-pull-secret-syncer-xsvnt" (UID: "15cde4c4-5e59-47e6-87f4-32de185a998e") : object "kube-system"/"original-pull-secret" not registered May 28 20:52:37.463450 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:37.463416 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:37.463641 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:37.463563 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:38.463342 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:38.463289 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:38.463342 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:38.463336 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:38.463832 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:38.463463 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:38.463832 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:38.463560 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:38.842794 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:38.842662 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:38.842950 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:38.842828 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 20:52:38.842950 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:38.842899 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret podName:15cde4c4-5e59-47e6-87f4-32de185a998e nodeName:}" failed. No retries permitted until 2026-05-28 20:52:42.842878855 +0000 UTC m=+12.960272422 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret") pod "global-pull-secret-syncer-xsvnt" (UID: "15cde4c4-5e59-47e6-87f4-32de185a998e") : object "kube-system"/"original-pull-secret" not registered May 28 20:52:39.463250 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:39.463218 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:39.463422 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:39.463367 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:40.054096 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:40.053536 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:40.054096 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:40.053699 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:40.054096 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:40.053762 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs podName:338d9053-7caa-48f3-bd43-6433cf054fc5 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:48.053742895 +0000 UTC m=+18.171136471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs") pod "network-metrics-daemon-qqths" (UID: "338d9053-7caa-48f3-bd43-6433cf054fc5") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:40.154507 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:40.154478 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gw6vb\" (UniqueName: \"kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb\") pod \"network-check-target-45r74\" (UID: \"5c11e3de-f375-4994-9fb5-26a89d3e0ec7\") " pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:40.154681 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:40.154622 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:52:40.154681 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:40.154653 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:52:40.154681 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:40.154665 2563 projected.go:194] Error preparing data for projected volume kube-api-access-gw6vb for pod openshift-network-diagnostics/network-check-target-45r74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:40.154851 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:40.154725 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb podName:5c11e3de-f375-4994-9fb5-26a89d3e0ec7 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:48.15470538 +0000 UTC m=+18.272098947 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw6vb" (UniqueName: "kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb") pod "network-check-target-45r74" (UID: "5c11e3de-f375-4994-9fb5-26a89d3e0ec7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:40.464400 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:40.464323 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:40.464828 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:40.464427 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:40.464903 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:40.464889 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:40.465020 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:40.464982 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:41.463732 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:41.463247 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:41.463732 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:41.463390 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:42.463405 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:42.463354 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:42.463835 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:42.463502 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:42.463835 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:42.463578 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:42.463835 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:42.463681 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:42.872476 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:42.872394 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:42.872648 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:42.872571 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 20:52:42.872648 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:42.872645 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret podName:15cde4c4-5e59-47e6-87f4-32de185a998e nodeName:}" failed. No retries permitted until 2026-05-28 20:52:50.872626232 +0000 UTC m=+20.990019810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret") pod "global-pull-secret-syncer-xsvnt" (UID: "15cde4c4-5e59-47e6-87f4-32de185a998e") : object "kube-system"/"original-pull-secret" not registered May 28 20:52:43.462864 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:43.462783 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:43.463002 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:43.462927 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:44.463636 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:44.463600 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:44.464093 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:44.463641 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:44.464093 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:44.463726 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:44.464093 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:44.463842 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:45.463661 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:45.463629 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:45.464070 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:45.463759 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:46.463198 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:46.463153 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:46.463379 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:46.463279 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:46.463379 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:46.463331 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:46.463490 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:46.463422 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:47.463332 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:47.463298 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:47.463838 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:47.463430 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:48.118308 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:48.118273 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:48.118516 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:48.118453 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:48.118583 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:48.118534 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs podName:338d9053-7caa-48f3-bd43-6433cf054fc5 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:04.118511658 +0000 UTC m=+34.235905222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs") pod "network-metrics-daemon-qqths" (UID: "338d9053-7caa-48f3-bd43-6433cf054fc5") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:48.219712 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:48.219671 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gw6vb\" (UniqueName: \"kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb\") pod \"network-check-target-45r74\" (UID: \"5c11e3de-f375-4994-9fb5-26a89d3e0ec7\") " pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:48.219893 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:48.219806 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:52:48.219893 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:48.219832 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:52:48.219893 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:48.219846 2563 projected.go:194] Error preparing data for projected volume kube-api-access-gw6vb for pod openshift-network-diagnostics/network-check-target-45r74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:48.220020 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:48.219918 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb podName:5c11e3de-f375-4994-9fb5-26a89d3e0ec7 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:04.219898946 +0000 UTC m=+34.337292532 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw6vb" (UniqueName: "kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb") pod "network-check-target-45r74" (UID: "5c11e3de-f375-4994-9fb5-26a89d3e0ec7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:48.463101 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:48.463026 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:48.463278 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:48.463026 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:48.463278 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:48.463128 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:48.463278 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:48.463248 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:49.462967 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:49.462929 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:49.463135 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:49.463062 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:50.463413 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:50.463387 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:50.463942 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:50.463496 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:50.463942 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:50.463569 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:50.463942 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:50.463650 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:50.940301 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:50.940235 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:50.940431 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:50.940340 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 20:52:50.940431 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:50.940390 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret podName:15cde4c4-5e59-47e6-87f4-32de185a998e nodeName:}" failed. No retries permitted until 2026-05-28 20:53:06.940376063 +0000 UTC m=+37.057769628 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret") pod "global-pull-secret-syncer-xsvnt" (UID: "15cde4c4-5e59-47e6-87f4-32de185a998e") : object "kube-system"/"original-pull-secret" not registered May 28 20:52:51.463383 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.463163 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:51.463502 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:51.463437 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:51.550012 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.549979 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" event={"ID":"40f60f87-8c1d-4092-96b5-d7ab2bcb1c68","Type":"ContainerStarted","Data":"30ab2563cbcba4d1ba6855ca00c87294d1b0342eee0cf64c43f159a9199f031a"} May 28 20:52:51.551202 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.551154 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l89rg" event={"ID":"9ac6fddb-e201-4648-8670-9a00e75dd19f","Type":"ContainerStarted","Data":"0198410832b8a9a99562941b9d42b02792820e0af73592fe1e488382741e079e"} May 28 20:52:51.552342 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.552322 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lkcmf" event={"ID":"946679ad-8718-48c0-98c2-03feb3f67b30","Type":"ContainerStarted","Data":"8403fb27034792e9be24478127e236d638e27083af0fbc547e35ba71249fae5f"} May 28 20:52:51.553504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.553480 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9lkdc" event={"ID":"943ced3a-27aa-4fb2-8ffd-53d1269faf80","Type":"ContainerStarted","Data":"3855ce1c421af55b0f4667f300eedae91dff4a147442614b5f780f564b12fb20"} May 28 20:52:51.554800 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.554777 2563 generic.go:358] "Generic (PLEG): container finished" podID="aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0" containerID="762f73c69fa0544a41e3e26ba3c5edc386f796cb63d52911d3cb7b1e0f9bf521" exitCode=0 May 28 20:52:51.554891 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.554838 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4tz8l" event={"ID":"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0","Type":"ContainerDied","Data":"762f73c69fa0544a41e3e26ba3c5edc386f796cb63d52911d3cb7b1e0f9bf521"} May 28 20:52:51.556092 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.556022 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" event={"ID":"c225cfeb-ded5-4744-989c-3cc9f0eb2136","Type":"ContainerStarted","Data":"b3baf8b00701d546a44a8ccc31d8d94c72e00855e7a8694465de12d00f1a01fc"} May 28 20:52:51.557213 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.557156 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2nxmw" event={"ID":"b289084f-e171-435e-8cf2-d5ba9965c060","Type":"ContainerStarted","Data":"ede4c7b36973ef484ad5931b1f0fb0b6e94dbf2a96cd97b4545f48c07652a296"} May 28 20:52:51.586780 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.586743 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kbnwf" podStartSLOduration=3.8775986529999997 podStartE2EDuration="21.586731352s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:52:33.32480207 +0000 UTC m=+3.442195634" lastFinishedPulling="2026-05-28 20:52:51.033934771 +0000 UTC m=+21.151328333" observedRunningTime="2026-05-28 20:52:51.586728435 +0000 UTC m=+21.704122020" watchObservedRunningTime="2026-05-28 20:52:51.586731352 +0000 UTC m=+21.704124954" May 28 20:52:51.586870 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.586854 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-94.ec2.internal" podStartSLOduration=20.586849861 podStartE2EDuration="20.586849861s" podCreationTimestamp="2026-05-28 20:52:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 20:52:35.538733099 +0000 UTC m=+5.656126686" watchObservedRunningTime="2026-05-28 20:52:51.586849861 +0000 UTC m=+21.704243445" May 28 20:52:51.616321 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.616284 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2nxmw" podStartSLOduration=3.978386158 podStartE2EDuration="21.616272875s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:52:33.333036697 +0000 UTC m=+3.450430260" lastFinishedPulling="2026-05-28 20:52:50.970923399 +0000 UTC m=+21.088316977" observedRunningTime="2026-05-28 20:52:51.616235759 +0000 UTC m=+21.733629356" watchObservedRunningTime="2026-05-28 20:52:51.616272875 +0000 UTC m=+21.733666460" May 28 20:52:51.682270 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.682232 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-l89rg" podStartSLOduration=3.9653384000000003 podStartE2EDuration="21.682221548s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:52:33.333804833 +0000 UTC m=+3.451198396" lastFinishedPulling="2026-05-28 20:52:51.050687977 +0000 UTC m=+21.168081544" observedRunningTime="2026-05-28 20:52:51.682107679 +0000 UTC m=+21.799501263" watchObservedRunningTime="2026-05-28 20:52:51.682221548 +0000 UTC m=+21.799615133" May 28 20:52:51.682416 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.682398 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-lkcmf" podStartSLOduration=3.969180091 podStartE2EDuration="21.682392766s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:52:33.326556864 +0000 UTC m=+3.443950427" lastFinishedPulling="2026-05-28 20:52:51.039769534 +0000 UTC m=+21.157163102" observedRunningTime="2026-05-28 20:52:51.644666707 +0000 UTC m=+21.762060292" watchObservedRunningTime="2026-05-28 20:52:51.682392766 +0000 UTC m=+21.799786351" May 28 20:52:51.771980 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:51.771941 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9lkdc" podStartSLOduration=3.104862603 podStartE2EDuration="20.77192768s" podCreationTimestamp="2026-05-28 20:52:31 +0000 UTC" firstStartedPulling="2026-05-28 20:52:33.32198404 +0000 UTC m=+3.439377606" lastFinishedPulling="2026-05-28 20:52:50.989049103 +0000 UTC m=+21.106442683" observedRunningTime="2026-05-28 20:52:51.707158792 +0000 UTC m=+21.824552377" watchObservedRunningTime="2026-05-28 20:52:51.77192768 +0000 UTC m=+21.889321264" May 28 20:52:52.463479 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:52.463318 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:52.463588 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:52.463347 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:52.463588 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:52.463567 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:52.464006 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:52.463664 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:52.560876 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:52.560664 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qsrfx" event={"ID":"0a02ff4d-d63e-44f7-b5c0-cf4c6c4e88e5","Type":"ContainerStarted","Data":"5106f24fb286713da4d9341d3551c0f562807061f00bec1f94bbc646522979d7"} May 28 20:52:52.562221 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:52.562191 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" event={"ID":"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8","Type":"ContainerStarted","Data":"5bcdabec3d7968ed0db7aaa041f7baf91b70573bee5222a5f9c7c75039bc565b"} May 28 20:52:52.579830 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:52.579803 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:52.594447 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:52.594404 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qsrfx" podStartSLOduration=4.890381931 podStartE2EDuration="22.594390986s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:52:33.32992638 +0000 UTC m=+3.447319951" lastFinishedPulling="2026-05-28 20:52:51.033935435 +0000 UTC m=+21.151329006" observedRunningTime="2026-05-28 20:52:52.593839357 +0000 UTC m=+22.711232945" watchObservedRunningTime="2026-05-28 20:52:52.594390986 +0000 UTC m=+22.711784549" May 28 20:52:52.746890 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:52.746776 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 28 20:52:53.455557 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.455454 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T20:52:52.74679214Z","UUID":"1e5a930d-c768-4ba0-991a-6951b97112b4","Handler":null,"Name":"","Endpoint":""} May 28 20:52:53.458321 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.458300 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 28 20:52:53.458321 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.458327 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 28 20:52:53.463342 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.463314 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:53.463472 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:53.463448 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:53.478420 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.478388 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:53.479039 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.479022 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:53.566113 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.566077 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" event={"ID":"c225cfeb-ded5-4744-989c-3cc9f0eb2136","Type":"ContainerStarted","Data":"850d10f10a28611b4cae1a74e1f23ee510d4ec90e13423a50623c817ff4535ca"} May 28 20:52:53.569450 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.569418 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" event={"ID":"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8","Type":"ContainerStarted","Data":"7aab42d675c2eb35a377aad65d976c707592c3f9abea9f238a10a39dfdc3629b"} May 28 20:52:53.569577 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.569455 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" event={"ID":"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8","Type":"ContainerStarted","Data":"9137bc387351fd1955bdf4d06db660755c0f46057988819734232dcb5bdfe1db"} May 28 20:52:53.569577 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.569464 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" event={"ID":"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8","Type":"ContainerStarted","Data":"98383790c7927368ab48e01389225c19f96a39ceda7da7a870dcd10748661c7a"} May 28 20:52:53.569577 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.569473 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" event={"ID":"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8","Type":"ContainerStarted","Data":"c9b30b4c4ce1f2531ad4a52acfce5fa41f4341bf3a88ceb4ff747d390c5e0ce1"} May 28 20:52:53.569577 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.569481 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" event={"ID":"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8","Type":"ContainerStarted","Data":"e35941a4f5813e65dc182616cc6c79ad80e9da28f6c389129b20f54e71057863"} May 28 20:52:53.570453 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:53.570366 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2nxmw" May 28 20:52:54.462898 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:54.462865 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:54.463111 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:54.462985 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:54.463111 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:54.463050 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:54.463250 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:54.463153 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:54.573566 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:54.573536 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" event={"ID":"c225cfeb-ded5-4744-989c-3cc9f0eb2136","Type":"ContainerStarted","Data":"22860d0d6dbe2350ff15563f76f4e63109f94bb6daa770dff579859da0a3c249"} May 28 20:52:54.600279 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:54.600238 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59kqk" podStartSLOduration=4.09379281 podStartE2EDuration="24.600225642s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:52:33.320605937 +0000 UTC m=+3.437999514" lastFinishedPulling="2026-05-28 20:52:53.827038782 +0000 UTC m=+23.944432346" observedRunningTime="2026-05-28 20:52:54.600144001 +0000 UTC m=+24.717537599" watchObservedRunningTime="2026-05-28 20:52:54.600225642 +0000 UTC m=+24.717619226" May 28 20:52:55.462982 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:55.462950 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:55.463137 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:55.463066 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:56.462823 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:56.462787 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:56.463404 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:56.462842 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:56.463404 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:56.462929 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:56.463404 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:56.462962 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:56.577490 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:56.577463 2563 generic.go:358] "Generic (PLEG): container finished" podID="aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0" containerID="bc302c29110d371bc6b941ecf3fe4df91cb7d0161193b6e84e5b96437ef35d6d" exitCode=0 May 28 20:52:56.577643 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:56.577545 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4tz8l" event={"ID":"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0","Type":"ContainerDied","Data":"bc302c29110d371bc6b941ecf3fe4df91cb7d0161193b6e84e5b96437ef35d6d"} May 28 20:52:56.580362 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:56.580339 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" event={"ID":"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8","Type":"ContainerStarted","Data":"b9bb2acf9c54b4f68358f7e35eb6e71aeaebbcb68b498059a29d130c5885cb59"} May 28 20:52:57.463459 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:57.463435 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:57.463877 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:57.463539 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:57.583604 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:57.583530 2563 generic.go:358] "Generic (PLEG): container finished" podID="aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0" containerID="0242b8a20338ec4bc7c8e8fe6d8635c0079eaee677f4c0bf4b71de26eea317ae" exitCode=0 May 28 20:52:57.583604 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:57.583592 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4tz8l" event={"ID":"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0","Type":"ContainerDied","Data":"0242b8a20338ec4bc7c8e8fe6d8635c0079eaee677f4c0bf4b71de26eea317ae"} May 28 20:52:58.462954 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:58.462774 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:58.463132 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:58.462774 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:58.463132 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:58.463026 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:58.463132 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:58.463109 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:52:58.586991 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:58.586908 2563 generic.go:358] "Generic (PLEG): container finished" podID="aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0" containerID="fb510f42d508187c10fd66070c7b6ce07418c54b01dabcd7982e96168f1b33f9" exitCode=0 May 28 20:52:58.587519 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:58.586997 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4tz8l" event={"ID":"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0","Type":"ContainerDied","Data":"fb510f42d508187c10fd66070c7b6ce07418c54b01dabcd7982e96168f1b33f9"} May 28 20:52:58.590110 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:58.590085 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" event={"ID":"e6f5ed42-ee6c-4a50-a715-93c3b4e92da8","Type":"ContainerStarted","Data":"397494c9878ce68af6d734cc3bd7f8690da95a1b747459e71b625e79681bd5fb"} May 28 20:52:58.590380 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:58.590365 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:58.590449 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:58.590390 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:58.590449 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:58.590402 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:58.605983 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:58.605957 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:58.606142 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:58.606041 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:52:59.462923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:59.462892 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:59.463089 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:59.463022 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:59.949762 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:59.949701 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" podStartSLOduration=11.008011409 podStartE2EDuration="29.949683623s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:52:33.329945156 +0000 UTC m=+3.447338718" lastFinishedPulling="2026-05-28 20:52:52.271617364 +0000 UTC m=+22.389010932" observedRunningTime="2026-05-28 20:52:58.659714679 +0000 UTC m=+28.777108264" watchObservedRunningTime="2026-05-28 20:52:59.949683623 +0000 UTC m=+30.067077209" May 28 20:52:59.950316 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:59.950296 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-45r74"] May 28 20:52:59.950444 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:59.950422 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:52:59.950559 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:59.950526 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:52:59.954462 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:59.954189 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qqths"] May 28 20:52:59.954462 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:59.954299 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:52:59.954462 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:59.954395 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:52:59.955083 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:59.954983 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xsvnt"] May 28 20:52:59.955138 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:52:59.955086 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:52:59.955208 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:52:59.955189 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:53:01.463329 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:01.463072 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:53:01.463763 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:01.463077 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:53:01.463763 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:01.463438 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:53:01.463763 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:01.463094 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:53:01.463763 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:01.463528 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:53:01.463763 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:01.463559 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:53:03.462939 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:03.462898 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:53:03.463459 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:03.462898 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:53:03.463459 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:03.463149 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:53:03.463459 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:03.462899 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:53:03.463459 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:03.463026 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:53:03.463459 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:03.463291 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:53:04.152912 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:04.152825 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:53:04.153052 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:04.153014 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:53:04.153095 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:04.153079 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs podName:338d9053-7caa-48f3-bd43-6433cf054fc5 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:36.153062872 +0000 UTC m=+66.270456436 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs") pod "network-metrics-daemon-qqths" (UID: "338d9053-7caa-48f3-bd43-6433cf054fc5") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:53:04.254198 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:04.254150 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gw6vb\" (UniqueName: \"kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb\") pod \"network-check-target-45r74\" (UID: \"5c11e3de-f375-4994-9fb5-26a89d3e0ec7\") " pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:53:04.254346 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:04.254303 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:53:04.254346 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:04.254322 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:53:04.254346 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:04.254331 2563 projected.go:194] Error preparing data for projected volume kube-api-access-gw6vb for pod openshift-network-diagnostics/network-check-target-45r74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:53:04.254445 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:04.254380 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb podName:5c11e3de-f375-4994-9fb5-26a89d3e0ec7 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:36.254365421 +0000 UTC m=+66.371758989 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gw6vb" (UniqueName: "kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb") pod "network-check-target-45r74" (UID: "5c11e3de-f375-4994-9fb5-26a89d3e0ec7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:53:05.462895 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:05.462867 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:53:05.463496 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:05.462870 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:53:05.463496 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:05.462968 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qqths" podUID="338d9053-7caa-48f3-bd43-6433cf054fc5" May 28 20:53:05.463496 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:05.463039 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45r74" podUID="5c11e3de-f375-4994-9fb5-26a89d3e0ec7" May 28 20:53:05.463496 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:05.462870 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:53:05.463496 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:05.463125 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xsvnt" podUID="15cde4c4-5e59-47e6-87f4-32de185a998e" May 28 20:53:05.605898 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:05.605869 2563 generic.go:358] "Generic (PLEG): container finished" podID="aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0" containerID="a0aaa63faa535a26065f7309e3c80d4eb78f03be6ef7dce983fe9c2eba770f53" exitCode=0 May 28 20:53:05.606037 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:05.605928 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4tz8l" event={"ID":"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0","Type":"ContainerDied","Data":"a0aaa63faa535a26065f7309e3c80d4eb78f03be6ef7dce983fe9c2eba770f53"} May 28 20:53:06.222730 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.222661 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-94.ec2.internal" event="NodeReady" May 28 20:53:06.222853 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.222775 2563 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 28 20:53:06.278526 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.278497 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h7jdr"] May 28 20:53:06.298654 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.298629 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s64pw"] May 28 20:53:06.298805 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.298788 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.304301 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.304025 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 28 20:53:06.304301 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.304054 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6mfq4\"" May 28 20:53:06.304602 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.304586 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 28 20:53:06.322672 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.322588 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h7jdr"] May 28 20:53:06.322787 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.322691 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s64pw"] May 28 20:53:06.322787 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.322697 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:06.324998 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.324978 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 28 20:53:06.325104 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.325053 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 28 20:53:06.325104 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.325053 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gngfj\"" May 28 20:53:06.325372 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.325357 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 28 20:53:06.369628 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.369594 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.369779 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.369642 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-config-volume\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.369779 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.369668 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s557\" (UniqueName: \"kubernetes.io/projected/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-kube-api-access-8s557\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.369779 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.369730 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcxsw\" (UniqueName: \"kubernetes.io/projected/87bd9d8d-23ac-400a-8c2f-b368202b054e-kube-api-access-tcxsw\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:06.369779 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.369761 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:06.369779 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.369777 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-tmp-dir\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.470298 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.470269 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tcxsw\" (UniqueName: \"kubernetes.io/projected/87bd9d8d-23ac-400a-8c2f-b368202b054e-kube-api-access-tcxsw\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:06.470298 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.470306 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:06.470777 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.470324 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-tmp-dir\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.470777 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.470350 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.470777 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.470388 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-config-volume\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.470777 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.470410 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8s557\" (UniqueName: \"kubernetes.io/projected/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-kube-api-access-8s557\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.470777 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:06.470413 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:53:06.470777 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:06.470552 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:53:06.470777 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:06.470557 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert podName:87bd9d8d-23ac-400a-8c2f-b368202b054e nodeName:}" failed. No retries permitted until 2026-05-28 20:53:06.970540075 +0000 UTC m=+37.087933645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert") pod "ingress-canary-s64pw" (UID: "87bd9d8d-23ac-400a-8c2f-b368202b054e") : secret "canary-serving-cert" not found May 28 20:53:06.470777 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:06.470624 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls podName:c81ee9a4-8670-4a9f-8326-e30cf38d0ee9 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:06.970605683 +0000 UTC m=+37.087999250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls") pod "dns-default-h7jdr" (UID: "c81ee9a4-8670-4a9f-8326-e30cf38d0ee9") : secret "dns-default-metrics-tls" not found May 28 20:53:06.470777 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.470698 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-tmp-dir\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.471131 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.471115 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-config-volume\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.486888 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.486865 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s557\" (UniqueName: \"kubernetes.io/projected/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-kube-api-access-8s557\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.487114 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.487092 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcxsw\" (UniqueName: \"kubernetes.io/projected/87bd9d8d-23ac-400a-8c2f-b368202b054e-kube-api-access-tcxsw\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:06.610918 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.610890 2563 generic.go:358] "Generic (PLEG): container finished" podID="aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0" containerID="c244d6d6ff97c19cb7ec2c339dbfdaa607f4de06a0f55c29e3f2346f75abdbd2" exitCode=0 May 28 20:53:06.611077 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.610938 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4tz8l" event={"ID":"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0","Type":"ContainerDied","Data":"c244d6d6ff97c19cb7ec2c339dbfdaa607f4de06a0f55c29e3f2346f75abdbd2"} May 28 20:53:06.974901 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.974722 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:06.975040 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.974918 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:06.975040 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:06.974940 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:53:06.975040 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:06.974871 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:53:06.975040 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:06.975030 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert podName:87bd9d8d-23ac-400a-8c2f-b368202b054e nodeName:}" failed. No retries permitted until 2026-05-28 20:53:07.975014291 +0000 UTC m=+38.092407859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert") pod "ingress-canary-s64pw" (UID: "87bd9d8d-23ac-400a-8c2f-b368202b054e") : secret "canary-serving-cert" not found May 28 20:53:06.975040 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:06.975036 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 20:53:06.975247 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:06.975040 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:53:06.975247 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:06.975087 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret podName:15cde4c4-5e59-47e6-87f4-32de185a998e nodeName:}" failed. No retries permitted until 2026-05-28 20:53:38.97507169 +0000 UTC m=+69.092465270 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret") pod "global-pull-secret-syncer-xsvnt" (UID: "15cde4c4-5e59-47e6-87f4-32de185a998e") : object "kube-system"/"original-pull-secret" not registered May 28 20:53:06.975247 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:06.975105 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls podName:c81ee9a4-8670-4a9f-8326-e30cf38d0ee9 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:07.975098964 +0000 UTC m=+38.092492526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls") pod "dns-default-h7jdr" (UID: "c81ee9a4-8670-4a9f-8326-e30cf38d0ee9") : secret "dns-default-metrics-tls" not found May 28 20:53:07.463631 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.463601 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:53:07.463799 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.463599 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:53:07.463866 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.463613 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:53:07.466823 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.466808 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 20:53:07.467215 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.467199 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 20:53:07.467295 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.467232 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kbvdl\"" May 28 20:53:07.467451 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.467435 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 28 20:53:07.467536 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.467522 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6jf7t\"" May 28 20:53:07.467626 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.467612 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 20:53:07.617562 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.617531 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4tz8l" event={"ID":"aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0","Type":"ContainerStarted","Data":"b32f0b738109bc4c587731cf8d00b6c7104411c7c2d76e2435b5f21c709d691d"} May 28 20:53:07.644569 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.644519 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4tz8l" podStartSLOduration=6.448589408 podStartE2EDuration="37.644505188s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:52:33.321983652 +0000 UTC m=+3.439377222" lastFinishedPulling="2026-05-28 20:53:04.517899436 +0000 UTC m=+34.635293002" observedRunningTime="2026-05-28 20:53:07.643160047 +0000 UTC m=+37.760553632" watchObservedRunningTime="2026-05-28 20:53:07.644505188 +0000 UTC m=+37.761898773" May 28 20:53:07.981164 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.981120 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:07.981337 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:07.981198 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:07.981337 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:07.981267 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:53:07.981337 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:07.981305 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:53:07.981337 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:07.981337 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert podName:87bd9d8d-23ac-400a-8c2f-b368202b054e nodeName:}" failed. No retries permitted until 2026-05-28 20:53:09.981320921 +0000 UTC m=+40.098714487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert") pod "ingress-canary-s64pw" (UID: "87bd9d8d-23ac-400a-8c2f-b368202b054e") : secret "canary-serving-cert" not found May 28 20:53:07.981496 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:07.981353 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls podName:c81ee9a4-8670-4a9f-8326-e30cf38d0ee9 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:09.981347314 +0000 UTC m=+40.098740877 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls") pod "dns-default-h7jdr" (UID: "c81ee9a4-8670-4a9f-8326-e30cf38d0ee9") : secret "dns-default-metrics-tls" not found May 28 20:53:09.995514 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:09.995480 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:09.995855 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:09.995573 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:09.995855 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:09.995633 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:53:09.995855 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:09.995645 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:53:09.995855 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:09.995694 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls podName:c81ee9a4-8670-4a9f-8326-e30cf38d0ee9 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:13.995679786 +0000 UTC m=+44.113073353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls") pod "dns-default-h7jdr" (UID: "c81ee9a4-8670-4a9f-8326-e30cf38d0ee9") : secret "dns-default-metrics-tls" not found May 28 20:53:09.995855 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:09.995707 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert podName:87bd9d8d-23ac-400a-8c2f-b368202b054e nodeName:}" failed. No retries permitted until 2026-05-28 20:53:13.995701629 +0000 UTC m=+44.113095206 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert") pod "ingress-canary-s64pw" (UID: "87bd9d8d-23ac-400a-8c2f-b368202b054e") : secret "canary-serving-cert" not found May 28 20:53:14.022397 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:14.022367 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:14.022854 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:14.022409 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:14.022854 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:14.022504 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:53:14.022854 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:14.022507 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:53:14.022854 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:14.022568 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert podName:87bd9d8d-23ac-400a-8c2f-b368202b054e nodeName:}" failed. No retries permitted until 2026-05-28 20:53:22.022553636 +0000 UTC m=+52.139947203 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert") pod "ingress-canary-s64pw" (UID: "87bd9d8d-23ac-400a-8c2f-b368202b054e") : secret "canary-serving-cert" not found May 28 20:53:14.022854 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:14.022583 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls podName:c81ee9a4-8670-4a9f-8326-e30cf38d0ee9 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:22.022576682 +0000 UTC m=+52.139970244 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls") pod "dns-default-h7jdr" (UID: "c81ee9a4-8670-4a9f-8326-e30cf38d0ee9") : secret "dns-default-metrics-tls" not found May 28 20:53:22.076441 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:22.076406 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:22.076902 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:22.076453 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:22.076902 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:22.076538 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:53:22.076902 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:22.076546 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:53:22.076902 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:22.076598 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls podName:c81ee9a4-8670-4a9f-8326-e30cf38d0ee9 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:38.076583115 +0000 UTC m=+68.193976681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls") pod "dns-default-h7jdr" (UID: "c81ee9a4-8670-4a9f-8326-e30cf38d0ee9") : secret "dns-default-metrics-tls" not found May 28 20:53:22.076902 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:22.076663 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert podName:87bd9d8d-23ac-400a-8c2f-b368202b054e nodeName:}" failed. No retries permitted until 2026-05-28 20:53:38.076648082 +0000 UTC m=+68.194041645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert") pod "ingress-canary-s64pw" (UID: "87bd9d8d-23ac-400a-8c2f-b368202b054e") : secret "canary-serving-cert" not found May 28 20:53:30.605748 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:30.605722 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlv9g" May 28 20:53:36.172634 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:36.172594 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:53:36.174919 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:36.174901 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 20:53:36.183622 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:36.183604 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 28 20:53:36.183678 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:36.183669 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs podName:338d9053-7caa-48f3-bd43-6433cf054fc5 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:40.183652515 +0000 UTC m=+130.301046079 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs") pod "network-metrics-daemon-qqths" (UID: "338d9053-7caa-48f3-bd43-6433cf054fc5") : secret "metrics-daemon-secret" not found May 28 20:53:36.273698 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:36.273673 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gw6vb\" (UniqueName: \"kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb\") pod \"network-check-target-45r74\" (UID: \"5c11e3de-f375-4994-9fb5-26a89d3e0ec7\") " pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:53:36.276098 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:36.276081 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 20:53:36.286762 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:36.286746 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 20:53:36.298205 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:36.298183 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw6vb\" (UniqueName: \"kubernetes.io/projected/5c11e3de-f375-4994-9fb5-26a89d3e0ec7-kube-api-access-gw6vb\") pod \"network-check-target-45r74\" (UID: \"5c11e3de-f375-4994-9fb5-26a89d3e0ec7\") " pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:53:36.581616 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:36.581581 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6jf7t\"" May 28 20:53:36.589270 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:36.589217 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:53:36.740657 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:36.740628 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-45r74"] May 28 20:53:36.744491 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:53:36.744468 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c11e3de_f375_4994_9fb5_26a89d3e0ec7.slice/crio-3b72de1ecd43caafb390a45144b2292c890a6e3ea7fcbb30dd96d3c8646cabb2 WatchSource:0}: Error finding container 3b72de1ecd43caafb390a45144b2292c890a6e3ea7fcbb30dd96d3c8646cabb2: Status 404 returned error can't find the container with id 3b72de1ecd43caafb390a45144b2292c890a6e3ea7fcbb30dd96d3c8646cabb2 May 28 20:53:37.678150 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:37.678117 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-45r74" event={"ID":"5c11e3de-f375-4994-9fb5-26a89d3e0ec7","Type":"ContainerStarted","Data":"3b72de1ecd43caafb390a45144b2292c890a6e3ea7fcbb30dd96d3c8646cabb2"} May 28 20:53:38.088238 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:38.088205 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:53:38.088428 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:38.088297 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:53:38.088428 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:38.088317 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:53:38.088428 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:38.088377 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls podName:c81ee9a4-8670-4a9f-8326-e30cf38d0ee9 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:10.088361934 +0000 UTC m=+100.205755520 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls") pod "dns-default-h7jdr" (UID: "c81ee9a4-8670-4a9f-8326-e30cf38d0ee9") : secret "dns-default-metrics-tls" not found May 28 20:53:38.088428 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:38.088394 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:53:38.088670 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:38.088440 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert podName:87bd9d8d-23ac-400a-8c2f-b368202b054e nodeName:}" failed. No retries permitted until 2026-05-28 20:54:10.088425285 +0000 UTC m=+100.205818850 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert") pod "ingress-canary-s64pw" (UID: "87bd9d8d-23ac-400a-8c2f-b368202b054e") : secret "canary-serving-cert" not found May 28 20:53:38.997662 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:38.997626 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:53:39.000297 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:39.000276 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 28 20:53:39.010798 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:39.010771 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15cde4c4-5e59-47e6-87f4-32de185a998e-original-pull-secret\") pod \"global-pull-secret-syncer-xsvnt\" (UID: \"15cde4c4-5e59-47e6-87f4-32de185a998e\") " pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:53:39.283929 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:39.283842 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xsvnt" May 28 20:53:39.730776 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:39.730749 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xsvnt"] May 28 20:53:39.734504 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:53:39.734476 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15cde4c4_5e59_47e6_87f4_32de185a998e.slice/crio-bfcde81d9d3181312e727d3fa06fae546bcb5d34d47565ab54a543f4a092f414 WatchSource:0}: Error finding container bfcde81d9d3181312e727d3fa06fae546bcb5d34d47565ab54a543f4a092f414: Status 404 returned error can't find the container with id bfcde81d9d3181312e727d3fa06fae546bcb5d34d47565ab54a543f4a092f414 May 28 20:53:40.685211 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:40.685157 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xsvnt" event={"ID":"15cde4c4-5e59-47e6-87f4-32de185a998e","Type":"ContainerStarted","Data":"bfcde81d9d3181312e727d3fa06fae546bcb5d34d47565ab54a543f4a092f414"} May 28 20:53:40.686574 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:40.686545 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-45r74" event={"ID":"5c11e3de-f375-4994-9fb5-26a89d3e0ec7","Type":"ContainerStarted","Data":"16a149113db3344011f7727bd97ba4db821bb8ff067526be14f8cfb1f6890937"} May 28 20:53:40.686798 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:40.686768 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:53:40.701767 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:40.701726 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-45r74" podStartSLOduration=67.804275586 podStartE2EDuration="1m10.701713203s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:53:36.746235773 +0000 UTC m=+66.863629336" lastFinishedPulling="2026-05-28 20:53:39.643673389 +0000 UTC m=+69.761066953" observedRunningTime="2026-05-28 20:53:40.701613507 +0000 UTC m=+70.819007094" watchObservedRunningTime="2026-05-28 20:53:40.701713203 +0000 UTC m=+70.819106768" May 28 20:53:43.693743 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:43.693650 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xsvnt" event={"ID":"15cde4c4-5e59-47e6-87f4-32de185a998e","Type":"ContainerStarted","Data":"fc947da28c4736688e97bd766d446355b5fd6aefc92a730fba1cde51a47faee9"} May 28 20:53:43.709494 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:43.709448 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xsvnt" podStartSLOduration=65.073357959 podStartE2EDuration="1m8.709435644s" podCreationTimestamp="2026-05-28 20:52:35 +0000 UTC" firstStartedPulling="2026-05-28 20:53:39.736135305 +0000 UTC m=+69.853528869" lastFinishedPulling="2026-05-28 20:53:43.372212983 +0000 UTC m=+73.489606554" observedRunningTime="2026-05-28 20:53:43.708693273 +0000 UTC m=+73.826086858" watchObservedRunningTime="2026-05-28 20:53:43.709435644 +0000 UTC m=+73.826829228" May 28 20:53:58.980692 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:58.980656 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6b5d5f9b5b-76hl9"] May 28 20:53:58.982812 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:58.982797 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:58.984680 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:58.984656 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-k5tvl\"" May 28 20:53:58.984796 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:58.984775 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" May 28 20:53:58.984858 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:58.984795 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 28 20:53:58.984858 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:58.984807 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" May 28 20:53:58.984957 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:58.984921 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 28 20:53:58.985188 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:58.985150 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" May 28 20:53:58.985497 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:58.985473 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" May 28 20:53:58.995455 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:58.995433 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b5d5f9b5b-76hl9"] May 28 20:53:59.089875 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.089847 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6"] May 28 20:53:59.091813 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.091794 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k"] May 28 20:53:59.091954 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.091935 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" May 28 20:53:59.093273 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.093256 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-ks28m"] May 28 20:53:59.093411 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.093396 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:53:59.094628 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.094606 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-ks28m" May 28 20:53:59.095545 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.095530 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" May 28 20:53:59.095634 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.095620 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" May 28 20:53:59.097062 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.097043 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" May 28 20:53:59.097162 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.097151 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" May 28 20:53:59.097274 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.097190 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7wrv4\"" May 28 20:53:59.097274 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.097215 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" May 28 20:53:59.097274 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.097153 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" May 28 20:53:59.097409 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.097354 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kcpcg\"" May 28 20:53:59.097409 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.097366 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8xmj4\"" May 28 20:53:59.097409 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.097377 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" May 28 20:53:59.097521 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.097414 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" May 28 20:53:59.097521 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.097492 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" May 28 20:53:59.111815 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.111797 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-ks28m"] May 28 20:53:59.112720 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.112702 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k"] May 28 20:53:59.115119 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.115103 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6"] May 28 20:53:59.127563 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.127542 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-stats-auth\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.127664 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.127574 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.127720 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.127674 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndbnb\" (UniqueName: \"kubernetes.io/projected/f9e058d4-ea36-4595-8372-5f2839212a4b-kube-api-access-ndbnb\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.127768 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.127737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-default-certificate\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.127768 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.127757 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.193347 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.193321 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9"] May 28 20:53:59.194883 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.194866 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" May 28 20:53:59.196724 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.196702 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" May 28 20:53:59.196896 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.196881 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" May 28 20:53:59.196971 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.196925 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" May 28 20:53:59.197155 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.197142 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" May 28 20:53:59.197260 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.197246 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qg664\"" May 28 20:53:59.205412 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.205389 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9"] May 28 20:53:59.228130 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.228108 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pqz4k\" (UID: \"53f63a50-8e76-44f3-b8db-c19090300d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:53:59.228242 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.228139 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ndbnb\" (UniqueName: \"kubernetes.io/projected/f9e058d4-ea36-4595-8372-5f2839212a4b-kube-api-access-ndbnb\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.228242 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.228158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4caaa6c-18e0-4e2b-a12c-1f596369b74a-config\") pod \"kube-storage-version-migrator-operator-565644f49-hrsg6\" (UID: \"d4caaa6c-18e0-4e2b-a12c-1f596369b74a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" May 28 20:53:59.228242 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.228207 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-default-certificate\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.228399 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.228250 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjrh2\" (UniqueName: \"kubernetes.io/projected/d4caaa6c-18e0-4e2b-a12c-1f596369b74a-kube-api-access-mjrh2\") pod \"kube-storage-version-migrator-operator-565644f49-hrsg6\" (UID: \"d4caaa6c-18e0-4e2b-a12c-1f596369b74a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" May 28 20:53:59.228399 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.228328 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.228399 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.228359 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-stats-auth\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.228399 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.228378 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4caaa6c-18e0-4e2b-a12c-1f596369b74a-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-hrsg6\" (UID: \"d4caaa6c-18e0-4e2b-a12c-1f596369b74a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" May 28 20:53:59.228572 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:59.228451 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 20:53:59.228572 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.228486 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkmvw\" (UniqueName: \"kubernetes.io/projected/53f63a50-8e76-44f3-b8db-c19090300d83-kube-api-access-jkmvw\") pod \"cluster-samples-operator-697587599-pqz4k\" (UID: \"53f63a50-8e76-44f3-b8db-c19090300d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:53:59.228572 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:59.228517 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs podName:f9e058d4-ea36-4595-8372-5f2839212a4b nodeName:}" failed. No retries permitted until 2026-05-28 20:53:59.728501186 +0000 UTC m=+89.845894752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs") pod "router-default-6b5d5f9b5b-76hl9" (UID: "f9e058d4-ea36-4595-8372-5f2839212a4b") : secret "router-metrics-certs-default" not found May 28 20:53:59.228572 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.228553 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.228742 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.228583 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lxzl\" (UniqueName: \"kubernetes.io/projected/81a8812c-d4bf-4281-8cca-7bde3ceb0dc3-kube-api-access-5lxzl\") pod \"volume-data-source-validator-5cf8c89d64-ks28m\" (UID: \"81a8812c-d4bf-4281-8cca-7bde3ceb0dc3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-ks28m" May 28 20:53:59.228742 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:59.228651 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle podName:f9e058d4-ea36-4595-8372-5f2839212a4b nodeName:}" failed. No retries permitted until 2026-05-28 20:53:59.7286334 +0000 UTC m=+89.846026980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle") pod "router-default-6b5d5f9b5b-76hl9" (UID: "f9e058d4-ea36-4595-8372-5f2839212a4b") : configmap references non-existent config key: service-ca.crt May 28 20:53:59.230613 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.230591 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-stats-auth\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.230786 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.230752 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-default-certificate\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.235826 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.235810 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndbnb\" (UniqueName: \"kubernetes.io/projected/f9e058d4-ea36-4595-8372-5f2839212a4b-kube-api-access-ndbnb\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.329102 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.329076 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pqz4k\" (UID: \"53f63a50-8e76-44f3-b8db-c19090300d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:53:59.329237 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.329109 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4caaa6c-18e0-4e2b-a12c-1f596369b74a-config\") pod \"kube-storage-version-migrator-operator-565644f49-hrsg6\" (UID: \"d4caaa6c-18e0-4e2b-a12c-1f596369b74a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" May 28 20:53:59.329237 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.329129 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsbjs\" (UniqueName: \"kubernetes.io/projected/ec7c2b71-2588-4c6a-93d0-be7df0291377-kube-api-access-rsbjs\") pod \"service-ca-operator-57b5b68b8-ppdw9\" (UID: \"ec7c2b71-2588-4c6a-93d0-be7df0291377\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" May 28 20:53:59.329237 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.329148 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ec7c2b71-2588-4c6a-93d0-be7df0291377-config\") pod \"service-ca-operator-57b5b68b8-ppdw9\" (UID: \"ec7c2b71-2588-4c6a-93d0-be7df0291377\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" May 28 20:53:59.329237 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.329192 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mjrh2\" (UniqueName: \"kubernetes.io/projected/d4caaa6c-18e0-4e2b-a12c-1f596369b74a-kube-api-access-mjrh2\") pod \"kube-storage-version-migrator-operator-565644f49-hrsg6\" (UID: \"d4caaa6c-18e0-4e2b-a12c-1f596369b74a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" May 28 20:53:59.329392 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:59.329239 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 28 20:53:59.329392 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:59.329332 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls podName:53f63a50-8e76-44f3-b8db-c19090300d83 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:59.829311798 +0000 UTC m=+89.946705376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls") pod "cluster-samples-operator-697587599-pqz4k" (UID: "53f63a50-8e76-44f3-b8db-c19090300d83") : secret "samples-operator-tls" not found May 28 20:53:59.329392 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.329366 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4caaa6c-18e0-4e2b-a12c-1f596369b74a-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-hrsg6\" (UID: \"d4caaa6c-18e0-4e2b-a12c-1f596369b74a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" May 28 20:53:59.329509 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.329392 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jkmvw\" (UniqueName: \"kubernetes.io/projected/53f63a50-8e76-44f3-b8db-c19090300d83-kube-api-access-jkmvw\") pod \"cluster-samples-operator-697587599-pqz4k\" (UID: \"53f63a50-8e76-44f3-b8db-c19090300d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:53:59.329509 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.329419 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec7c2b71-2588-4c6a-93d0-be7df0291377-serving-cert\") pod \"service-ca-operator-57b5b68b8-ppdw9\" (UID: \"ec7c2b71-2588-4c6a-93d0-be7df0291377\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" May 28 20:53:59.329608 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.329599 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5lxzl\" (UniqueName: \"kubernetes.io/projected/81a8812c-d4bf-4281-8cca-7bde3ceb0dc3-kube-api-access-5lxzl\") pod \"volume-data-source-validator-5cf8c89d64-ks28m\" (UID: \"81a8812c-d4bf-4281-8cca-7bde3ceb0dc3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-ks28m" May 28 20:53:59.329659 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.329642 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d4caaa6c-18e0-4e2b-a12c-1f596369b74a-config\") pod \"kube-storage-version-migrator-operator-565644f49-hrsg6\" (UID: \"d4caaa6c-18e0-4e2b-a12c-1f596369b74a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" May 28 20:53:59.331417 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.331398 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d4caaa6c-18e0-4e2b-a12c-1f596369b74a-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-hrsg6\" (UID: \"d4caaa6c-18e0-4e2b-a12c-1f596369b74a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" May 28 20:53:59.337822 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.337803 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjrh2\" (UniqueName: \"kubernetes.io/projected/d4caaa6c-18e0-4e2b-a12c-1f596369b74a-kube-api-access-mjrh2\") pod \"kube-storage-version-migrator-operator-565644f49-hrsg6\" (UID: \"d4caaa6c-18e0-4e2b-a12c-1f596369b74a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" May 28 20:53:59.338233 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.338214 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lxzl\" (UniqueName: \"kubernetes.io/projected/81a8812c-d4bf-4281-8cca-7bde3ceb0dc3-kube-api-access-5lxzl\") pod \"volume-data-source-validator-5cf8c89d64-ks28m\" (UID: \"81a8812c-d4bf-4281-8cca-7bde3ceb0dc3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-ks28m" May 28 20:53:59.338320 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.338280 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkmvw\" (UniqueName: \"kubernetes.io/projected/53f63a50-8e76-44f3-b8db-c19090300d83-kube-api-access-jkmvw\") pod \"cluster-samples-operator-697587599-pqz4k\" (UID: \"53f63a50-8e76-44f3-b8db-c19090300d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:53:59.401455 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.401430 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" May 28 20:53:59.412313 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.412295 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-ks28m" May 28 20:53:59.430072 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.430043 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ec7c2b71-2588-4c6a-93d0-be7df0291377-config\") pod \"service-ca-operator-57b5b68b8-ppdw9\" (UID: \"ec7c2b71-2588-4c6a-93d0-be7df0291377\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" May 28 20:53:59.430907 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.430157 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec7c2b71-2588-4c6a-93d0-be7df0291377-serving-cert\") pod \"service-ca-operator-57b5b68b8-ppdw9\" (UID: \"ec7c2b71-2588-4c6a-93d0-be7df0291377\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" May 28 20:53:59.430907 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.430663 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ec7c2b71-2588-4c6a-93d0-be7df0291377-config\") pod \"service-ca-operator-57b5b68b8-ppdw9\" (UID: \"ec7c2b71-2588-4c6a-93d0-be7df0291377\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" May 28 20:53:59.430907 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.430746 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rsbjs\" (UniqueName: \"kubernetes.io/projected/ec7c2b71-2588-4c6a-93d0-be7df0291377-kube-api-access-rsbjs\") pod \"service-ca-operator-57b5b68b8-ppdw9\" (UID: \"ec7c2b71-2588-4c6a-93d0-be7df0291377\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" May 28 20:53:59.432658 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.432639 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec7c2b71-2588-4c6a-93d0-be7df0291377-serving-cert\") pod \"service-ca-operator-57b5b68b8-ppdw9\" (UID: \"ec7c2b71-2588-4c6a-93d0-be7df0291377\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" May 28 20:53:59.438360 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.438339 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsbjs\" (UniqueName: \"kubernetes.io/projected/ec7c2b71-2588-4c6a-93d0-be7df0291377-kube-api-access-rsbjs\") pod \"service-ca-operator-57b5b68b8-ppdw9\" (UID: \"ec7c2b71-2588-4c6a-93d0-be7df0291377\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" May 28 20:53:59.503440 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.503414 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" May 28 20:53:59.535192 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.534686 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6"] May 28 20:53:59.539423 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:53:59.539384 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4caaa6c_18e0_4e2b_a12c_1f596369b74a.slice/crio-71933ce7801af2efec17ab44a1eefbd7c2071ca77ccd27c3c4ae1bdb6af9ad3b WatchSource:0}: Error finding container 71933ce7801af2efec17ab44a1eefbd7c2071ca77ccd27c3c4ae1bdb6af9ad3b: Status 404 returned error can't find the container with id 71933ce7801af2efec17ab44a1eefbd7c2071ca77ccd27c3c4ae1bdb6af9ad3b May 28 20:53:59.553058 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.553031 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-ks28m"] May 28 20:53:59.558490 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:53:59.558456 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a8812c_d4bf_4281_8cca_7bde3ceb0dc3.slice/crio-48767c1db359d7642d3238d2ced716a3617b01022ce50b69ffbb5da3138d5662 WatchSource:0}: Error finding container 48767c1db359d7642d3238d2ced716a3617b01022ce50b69ffbb5da3138d5662: Status 404 returned error can't find the container with id 48767c1db359d7642d3238d2ced716a3617b01022ce50b69ffbb5da3138d5662 May 28 20:53:59.630417 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.630387 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9"] May 28 20:53:59.633747 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:53:59.633711 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec7c2b71_2588_4c6a_93d0_be7df0291377.slice/crio-59c1cd2b114cc83f64df0cb0271a9da148b486beb424228c58c6edb506e0c7cd WatchSource:0}: Error finding container 59c1cd2b114cc83f64df0cb0271a9da148b486beb424228c58c6edb506e0c7cd: Status 404 returned error can't find the container with id 59c1cd2b114cc83f64df0cb0271a9da148b486beb424228c58c6edb506e0c7cd May 28 20:53:59.724917 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.724880 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" event={"ID":"ec7c2b71-2588-4c6a-93d0-be7df0291377","Type":"ContainerStarted","Data":"59c1cd2b114cc83f64df0cb0271a9da148b486beb424228c58c6edb506e0c7cd"} May 28 20:53:59.725833 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.725789 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-ks28m" event={"ID":"81a8812c-d4bf-4281-8cca-7bde3ceb0dc3","Type":"ContainerStarted","Data":"48767c1db359d7642d3238d2ced716a3617b01022ce50b69ffbb5da3138d5662"} May 28 20:53:59.726680 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.726659 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" event={"ID":"d4caaa6c-18e0-4e2b-a12c-1f596369b74a","Type":"ContainerStarted","Data":"71933ce7801af2efec17ab44a1eefbd7c2071ca77ccd27c3c4ae1bdb6af9ad3b"} May 28 20:53:59.733211 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.733191 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.733308 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.733240 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:53:59.733365 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:59.733339 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 20:53:59.733407 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:59.733394 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs podName:f9e058d4-ea36-4595-8372-5f2839212a4b nodeName:}" failed. No retries permitted until 2026-05-28 20:54:00.733380989 +0000 UTC m=+90.850774557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs") pod "router-default-6b5d5f9b5b-76hl9" (UID: "f9e058d4-ea36-4595-8372-5f2839212a4b") : secret "router-metrics-certs-default" not found May 28 20:53:59.733450 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:59.733407 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle podName:f9e058d4-ea36-4595-8372-5f2839212a4b nodeName:}" failed. No retries permitted until 2026-05-28 20:54:00.733401545 +0000 UTC m=+90.850795107 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle") pod "router-default-6b5d5f9b5b-76hl9" (UID: "f9e058d4-ea36-4595-8372-5f2839212a4b") : configmap references non-existent config key: service-ca.crt May 28 20:53:59.833917 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:53:59.833823 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pqz4k\" (UID: \"53f63a50-8e76-44f3-b8db-c19090300d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:53:59.834071 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:59.833945 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 28 20:53:59.834071 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:53:59.834014 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls podName:53f63a50-8e76-44f3-b8db-c19090300d83 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:00.833993728 +0000 UTC m=+90.951387305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls") pod "cluster-samples-operator-697587599-pqz4k" (UID: "53f63a50-8e76-44f3-b8db-c19090300d83") : secret "samples-operator-tls" not found May 28 20:54:00.741733 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:00.741689 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:00.742132 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:00.741750 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:00.742132 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:00.741847 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 20:54:00.742132 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:00.741921 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs podName:f9e058d4-ea36-4595-8372-5f2839212a4b nodeName:}" failed. No retries permitted until 2026-05-28 20:54:02.741900589 +0000 UTC m=+92.859294157 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs") pod "router-default-6b5d5f9b5b-76hl9" (UID: "f9e058d4-ea36-4595-8372-5f2839212a4b") : secret "router-metrics-certs-default" not found May 28 20:54:00.742132 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:00.741942 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle podName:f9e058d4-ea36-4595-8372-5f2839212a4b nodeName:}" failed. No retries permitted until 2026-05-28 20:54:02.741930379 +0000 UTC m=+92.859323946 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle") pod "router-default-6b5d5f9b5b-76hl9" (UID: "f9e058d4-ea36-4595-8372-5f2839212a4b") : configmap references non-existent config key: service-ca.crt May 28 20:54:00.842435 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:00.842397 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pqz4k\" (UID: \"53f63a50-8e76-44f3-b8db-c19090300d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:54:00.842613 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:00.842560 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 28 20:54:00.842681 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:00.842648 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls podName:53f63a50-8e76-44f3-b8db-c19090300d83 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:02.84261985 +0000 UTC m=+92.960013416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls") pod "cluster-samples-operator-697587599-pqz4k" (UID: "53f63a50-8e76-44f3-b8db-c19090300d83") : secret "samples-operator-tls" not found May 28 20:54:02.733808 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:02.733766 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" event={"ID":"ec7c2b71-2588-4c6a-93d0-be7df0291377","Type":"ContainerStarted","Data":"8054b3f8d842fe2114cb2fa655d1b27bd18fa26f131d47a6ffd6215636f085ab"} May 28 20:54:02.735159 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:02.735134 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-ks28m" event={"ID":"81a8812c-d4bf-4281-8cca-7bde3ceb0dc3","Type":"ContainerStarted","Data":"9204f6f54f7508c142553f791a400589f5809a9ade3240407de4d1178a0b94f3"} May 28 20:54:02.736452 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:02.736428 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" event={"ID":"d4caaa6c-18e0-4e2b-a12c-1f596369b74a","Type":"ContainerStarted","Data":"167aaab911f160811060a6400adb926c6e288c49a7a6632805b1da327867a375"} May 28 20:54:02.749657 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:02.749606 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" podStartSLOduration=1.5506847910000001 podStartE2EDuration="3.749591012s" podCreationTimestamp="2026-05-28 20:53:59 +0000 UTC" firstStartedPulling="2026-05-28 20:53:59.635577742 +0000 UTC m=+89.752971306" lastFinishedPulling="2026-05-28 20:54:01.834483963 +0000 UTC m=+91.951877527" observedRunningTime="2026-05-28 20:54:02.748824946 +0000 UTC m=+92.866218533" watchObservedRunningTime="2026-05-28 20:54:02.749591012 +0000 UTC m=+92.866984599" May 28 20:54:02.761385 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:02.761362 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:02.761565 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:02.761416 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:02.761565 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:02.761508 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 20:54:02.761565 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:02.761566 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs podName:f9e058d4-ea36-4595-8372-5f2839212a4b nodeName:}" failed. No retries permitted until 2026-05-28 20:54:06.761549385 +0000 UTC m=+96.878942956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs") pod "router-default-6b5d5f9b5b-76hl9" (UID: "f9e058d4-ea36-4595-8372-5f2839212a4b") : secret "router-metrics-certs-default" not found May 28 20:54:02.761767 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:02.761583 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle podName:f9e058d4-ea36-4595-8372-5f2839212a4b nodeName:}" failed. No retries permitted until 2026-05-28 20:54:06.761574575 +0000 UTC m=+96.878968137 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle") pod "router-default-6b5d5f9b5b-76hl9" (UID: "f9e058d4-ea36-4595-8372-5f2839212a4b") : configmap references non-existent config key: service-ca.crt May 28 20:54:02.762931 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:02.762887 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" podStartSLOduration=1.472318934 podStartE2EDuration="3.762871133s" podCreationTimestamp="2026-05-28 20:53:59 +0000 UTC" firstStartedPulling="2026-05-28 20:53:59.541282466 +0000 UTC m=+89.658676029" lastFinishedPulling="2026-05-28 20:54:01.831834663 +0000 UTC m=+91.949228228" observedRunningTime="2026-05-28 20:54:02.762023256 +0000 UTC m=+92.879416842" watchObservedRunningTime="2026-05-28 20:54:02.762871133 +0000 UTC m=+92.880264720" May 28 20:54:02.786458 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:02.786408 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-ks28m" podStartSLOduration=1.5186572680000001 podStartE2EDuration="3.786392267s" podCreationTimestamp="2026-05-28 20:53:59 +0000 UTC" firstStartedPulling="2026-05-28 20:53:59.560417529 +0000 UTC m=+89.677811106" lastFinishedPulling="2026-05-28 20:54:01.828152527 +0000 UTC m=+91.945546105" observedRunningTime="2026-05-28 20:54:02.786050591 +0000 UTC m=+92.903444177" watchObservedRunningTime="2026-05-28 20:54:02.786392267 +0000 UTC m=+92.903785853" May 28 20:54:02.862844 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:02.862763 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pqz4k\" (UID: \"53f63a50-8e76-44f3-b8db-c19090300d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:54:02.863021 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:02.862882 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 28 20:54:02.863021 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:02.862959 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls podName:53f63a50-8e76-44f3-b8db-c19090300d83 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:06.862938069 +0000 UTC m=+96.980331643 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls") pod "cluster-samples-operator-697587599-pqz4k" (UID: "53f63a50-8e76-44f3-b8db-c19090300d83") : secret "samples-operator-tls" not found May 28 20:54:03.955377 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:03.955328 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz"] May 28 20:54:03.958521 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:03.958500 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz" May 28 20:54:03.960739 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:03.960719 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" May 28 20:54:03.961299 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:03.961284 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" May 28 20:54:03.961299 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:03.961289 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ckgvh\"" May 28 20:54:03.969658 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:03.969633 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz"] May 28 20:54:04.074038 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:04.074006 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf4bv\" (UniqueName: \"kubernetes.io/projected/f46c90a9-c203-4962-acd5-5fe69b70180b-kube-api-access-zf4bv\") pod \"migrator-b985f5d89-fndzz\" (UID: \"f46c90a9-c203-4962-acd5-5fe69b70180b\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz" May 28 20:54:04.175203 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:04.175153 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zf4bv\" (UniqueName: \"kubernetes.io/projected/f46c90a9-c203-4962-acd5-5fe69b70180b-kube-api-access-zf4bv\") pod \"migrator-b985f5d89-fndzz\" (UID: \"f46c90a9-c203-4962-acd5-5fe69b70180b\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz" May 28 20:54:04.183030 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:04.183002 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf4bv\" (UniqueName: \"kubernetes.io/projected/f46c90a9-c203-4962-acd5-5fe69b70180b-kube-api-access-zf4bv\") pod \"migrator-b985f5d89-fndzz\" (UID: \"f46c90a9-c203-4962-acd5-5fe69b70180b\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz" May 28 20:54:04.267731 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:04.267684 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz" May 28 20:54:04.383852 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:04.383820 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz"] May 28 20:54:04.387909 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:54:04.387883 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf46c90a9_c203_4962_acd5_5fe69b70180b.slice/crio-18559fc758f8f49dcdfdfd6748a6f4926a1770072aa340f9fc6f235f09929e81 WatchSource:0}: Error finding container 18559fc758f8f49dcdfdfd6748a6f4926a1770072aa340f9fc6f235f09929e81: Status 404 returned error can't find the container with id 18559fc758f8f49dcdfdfd6748a6f4926a1770072aa340f9fc6f235f09929e81 May 28 20:54:04.742050 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:04.742014 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz" event={"ID":"f46c90a9-c203-4962-acd5-5fe69b70180b","Type":"ContainerStarted","Data":"18559fc758f8f49dcdfdfd6748a6f4926a1770072aa340f9fc6f235f09929e81"} May 28 20:54:05.068814 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:05.068741 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lkcmf_946679ad-8718-48c0-98c2-03feb3f67b30/dns-node-resolver/0.log" May 28 20:54:05.746153 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:05.746122 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz" event={"ID":"f46c90a9-c203-4962-acd5-5fe69b70180b","Type":"ContainerStarted","Data":"26ba1e068ebc1d7bf8b6cc0c8f9387f27d0a782e22d51693b9e9de7a099aa9cd"} May 28 20:54:06.069729 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:06.069700 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9lkdc_943ced3a-27aa-4fb2-8ffd-53d1269faf80/node-ca/0.log" May 28 20:54:06.750601 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:06.750568 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz" event={"ID":"f46c90a9-c203-4962-acd5-5fe69b70180b","Type":"ContainerStarted","Data":"66d016b8056425aaa6e649985292aeba59c353083fbd5064ee4098d376e82262"} May 28 20:54:06.797790 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:06.797764 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:06.797945 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:06.797802 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:06.797945 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:06.797900 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 28 20:54:06.797945 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:06.797913 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle podName:f9e058d4-ea36-4595-8372-5f2839212a4b nodeName:}" failed. No retries permitted until 2026-05-28 20:54:14.797899741 +0000 UTC m=+104.915293304 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle") pod "router-default-6b5d5f9b5b-76hl9" (UID: "f9e058d4-ea36-4595-8372-5f2839212a4b") : configmap references non-existent config key: service-ca.crt May 28 20:54:06.798053 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:06.797947 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs podName:f9e058d4-ea36-4595-8372-5f2839212a4b nodeName:}" failed. No retries permitted until 2026-05-28 20:54:14.797932702 +0000 UTC m=+104.915326270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs") pod "router-default-6b5d5f9b5b-76hl9" (UID: "f9e058d4-ea36-4595-8372-5f2839212a4b") : secret "router-metrics-certs-default" not found May 28 20:54:06.898388 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:06.898357 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pqz4k\" (UID: \"53f63a50-8e76-44f3-b8db-c19090300d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:54:06.898544 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:06.898522 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 28 20:54:06.898607 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:06.898597 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls podName:53f63a50-8e76-44f3-b8db-c19090300d83 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:14.898579174 +0000 UTC m=+105.015972737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls") pod "cluster-samples-operator-697587599-pqz4k" (UID: "53f63a50-8e76-44f3-b8db-c19090300d83") : secret "samples-operator-tls" not found May 28 20:54:10.123162 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:10.123111 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:54:10.123674 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:10.123206 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:54:10.123674 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:10.123287 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:54:10.123674 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:10.123370 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert podName:87bd9d8d-23ac-400a-8c2f-b368202b054e nodeName:}" failed. No retries permitted until 2026-05-28 20:55:14.123347455 +0000 UTC m=+164.240741024 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert") pod "ingress-canary-s64pw" (UID: "87bd9d8d-23ac-400a-8c2f-b368202b054e") : secret "canary-serving-cert" not found May 28 20:54:10.123674 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:10.123296 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:54:10.123674 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:10.123420 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls podName:c81ee9a4-8670-4a9f-8326-e30cf38d0ee9 nodeName:}" failed. No retries permitted until 2026-05-28 20:55:14.123408076 +0000 UTC m=+164.240801639 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls") pod "dns-default-h7jdr" (UID: "c81ee9a4-8670-4a9f-8326-e30cf38d0ee9") : secret "dns-default-metrics-tls" not found May 28 20:54:11.691065 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:11.691035 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-45r74" May 28 20:54:11.706845 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:11.706803 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-fndzz" podStartSLOduration=7.424517507 podStartE2EDuration="8.706788912s" podCreationTimestamp="2026-05-28 20:54:03 +0000 UTC" firstStartedPulling="2026-05-28 20:54:04.389830381 +0000 UTC m=+94.507223951" lastFinishedPulling="2026-05-28 20:54:05.672101793 +0000 UTC m=+95.789495356" observedRunningTime="2026-05-28 20:54:06.770044145 +0000 UTC m=+96.887437729" watchObservedRunningTime="2026-05-28 20:54:11.706788912 +0000 UTC m=+101.824182497" May 28 20:54:14.864297 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:14.864263 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:14.864659 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:14.864319 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:14.864659 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:14.864513 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle podName:f9e058d4-ea36-4595-8372-5f2839212a4b nodeName:}" failed. No retries permitted until 2026-05-28 20:54:30.864496605 +0000 UTC m=+120.981890168 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle") pod "router-default-6b5d5f9b5b-76hl9" (UID: "f9e058d4-ea36-4595-8372-5f2839212a4b") : configmap references non-existent config key: service-ca.crt May 28 20:54:14.866778 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:14.866754 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9e058d4-ea36-4595-8372-5f2839212a4b-metrics-certs\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:14.965689 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:14.965654 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pqz4k\" (UID: \"53f63a50-8e76-44f3-b8db-c19090300d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:54:14.967902 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:14.967880 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53f63a50-8e76-44f3-b8db-c19090300d83-samples-operator-tls\") pod \"cluster-samples-operator-697587599-pqz4k\" (UID: \"53f63a50-8e76-44f3-b8db-c19090300d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:54:15.007739 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:15.007713 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" May 28 20:54:15.125668 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:15.125596 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k"] May 28 20:54:15.774541 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:15.774507 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" event={"ID":"53f63a50-8e76-44f3-b8db-c19090300d83","Type":"ContainerStarted","Data":"3ac76b9fcfba4a5e1209b8da3c393fe7f80a9c7a93ecec18707f1ccd23debc45"} May 28 20:54:17.782561 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:17.782533 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" event={"ID":"53f63a50-8e76-44f3-b8db-c19090300d83","Type":"ContainerStarted","Data":"13e51f36c7f605211658064a0fdb35b1639036cfab418354831326b6cf5de968"} May 28 20:54:17.782935 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:17.782566 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" event={"ID":"53f63a50-8e76-44f3-b8db-c19090300d83","Type":"ContainerStarted","Data":"37dfaf9ebbc8d71fbc5fe2bc6df7352cf719f12cb8c0c5262436cac5b3e32f7e"} May 28 20:54:17.797877 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:17.797838 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-pqz4k" podStartSLOduration=16.718277762 podStartE2EDuration="18.797825747s" podCreationTimestamp="2026-05-28 20:53:59 +0000 UTC" firstStartedPulling="2026-05-28 20:54:15.198097499 +0000 UTC m=+105.315491062" lastFinishedPulling="2026-05-28 20:54:17.277645484 +0000 UTC m=+107.395039047" observedRunningTime="2026-05-28 20:54:17.797580596 +0000 UTC m=+107.914974183" watchObservedRunningTime="2026-05-28 20:54:17.797825747 +0000 UTC m=+107.915219331" May 28 20:54:26.236839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.236803 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dpgcb"] May 28 20:54:26.239816 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.239797 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.241881 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.241858 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 28 20:54:26.241987 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.241884 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 28 20:54:26.242550 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.242531 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gc8h7\"" May 28 20:54:26.242698 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.242666 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 28 20:54:26.242837 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.242614 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 28 20:54:26.248312 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.248293 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5d326dd-b9d0-4249-a11e-fb2f4829b875-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.248399 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.248361 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5d326dd-b9d0-4249-a11e-fb2f4829b875-crio-socket\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.248399 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.248380 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5d326dd-b9d0-4249-a11e-fb2f4829b875-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.248486 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.248406 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqnzs\" (UniqueName: \"kubernetes.io/projected/f5d326dd-b9d0-4249-a11e-fb2f4829b875-kube-api-access-xqnzs\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.248486 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.248434 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5d326dd-b9d0-4249-a11e-fb2f4829b875-data-volume\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.251183 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.251150 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dpgcb"] May 28 20:54:26.348820 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.348787 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xqnzs\" (UniqueName: \"kubernetes.io/projected/f5d326dd-b9d0-4249-a11e-fb2f4829b875-kube-api-access-xqnzs\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.348983 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.348849 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5d326dd-b9d0-4249-a11e-fb2f4829b875-data-volume\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.348983 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.348883 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5d326dd-b9d0-4249-a11e-fb2f4829b875-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.348983 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.348954 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5d326dd-b9d0-4249-a11e-fb2f4829b875-crio-socket\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.349116 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.348983 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5d326dd-b9d0-4249-a11e-fb2f4829b875-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.349116 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.349077 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5d326dd-b9d0-4249-a11e-fb2f4829b875-crio-socket\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.349344 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.349326 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5d326dd-b9d0-4249-a11e-fb2f4829b875-data-volume\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.349516 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.349492 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5d326dd-b9d0-4249-a11e-fb2f4829b875-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.351300 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.351284 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5d326dd-b9d0-4249-a11e-fb2f4829b875-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.357067 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.357047 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqnzs\" (UniqueName: \"kubernetes.io/projected/f5d326dd-b9d0-4249-a11e-fb2f4829b875-kube-api-access-xqnzs\") pod \"insights-runtime-extractor-dpgcb\" (UID: \"f5d326dd-b9d0-4249-a11e-fb2f4829b875\") " pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.548496 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.548402 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dpgcb" May 28 20:54:26.666396 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.666361 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dpgcb"] May 28 20:54:26.669633 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:54:26.669604 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5d326dd_b9d0_4249_a11e_fb2f4829b875.slice/crio-c7f76d4b191aaa12f15b3fd87eb5eec6bc38da3b35dbe5e08762d14231455cf7 WatchSource:0}: Error finding container c7f76d4b191aaa12f15b3fd87eb5eec6bc38da3b35dbe5e08762d14231455cf7: Status 404 returned error can't find the container with id c7f76d4b191aaa12f15b3fd87eb5eec6bc38da3b35dbe5e08762d14231455cf7 May 28 20:54:26.805408 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.805330 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpgcb" event={"ID":"f5d326dd-b9d0-4249-a11e-fb2f4829b875","Type":"ContainerStarted","Data":"917d2bf0f1611d5011975b50b3fae08d660d5ccee8f990dc4b6281d47945b606"} May 28 20:54:26.805408 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:26.805365 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpgcb" event={"ID":"f5d326dd-b9d0-4249-a11e-fb2f4829b875","Type":"ContainerStarted","Data":"c7f76d4b191aaa12f15b3fd87eb5eec6bc38da3b35dbe5e08762d14231455cf7"} May 28 20:54:27.810425 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:27.810386 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpgcb" event={"ID":"f5d326dd-b9d0-4249-a11e-fb2f4829b875","Type":"ContainerStarted","Data":"8569484366b14448d598bf10cf892f3d7155a05a8a6e5f3f471058cfba91a8b3"} May 28 20:54:29.820609 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:29.820573 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpgcb" event={"ID":"f5d326dd-b9d0-4249-a11e-fb2f4829b875","Type":"ContainerStarted","Data":"517a8b30006de869d880e696fc2b9f0a83c248b4614919dfa6d903c1d1d6a4a1"} May 28 20:54:29.837135 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:29.837088 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dpgcb" podStartSLOduration=1.6547499700000001 podStartE2EDuration="3.837075244s" podCreationTimestamp="2026-05-28 20:54:26 +0000 UTC" firstStartedPulling="2026-05-28 20:54:26.722410019 +0000 UTC m=+116.839803587" lastFinishedPulling="2026-05-28 20:54:28.904735298 +0000 UTC m=+119.022128861" observedRunningTime="2026-05-28 20:54:29.836644583 +0000 UTC m=+119.954038159" watchObservedRunningTime="2026-05-28 20:54:29.837075244 +0000 UTC m=+119.954468828" May 28 20:54:30.878256 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:30.878216 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:30.878816 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:30.878796 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9e058d4-ea36-4595-8372-5f2839212a4b-service-ca-bundle\") pod \"router-default-6b5d5f9b5b-76hl9\" (UID: \"f9e058d4-ea36-4595-8372-5f2839212a4b\") " pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:31.095588 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:31.095563 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-k5tvl\"" May 28 20:54:31.104088 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:31.104060 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:31.225183 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:31.225143 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b5d5f9b5b-76hl9"] May 28 20:54:31.227979 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:54:31.227950 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e058d4_ea36_4595_8372_5f2839212a4b.slice/crio-4e2aecee09cf5bd3f91d3d60dd1e649158e7fab908bda2bb30604f1ed8c08f37 WatchSource:0}: Error finding container 4e2aecee09cf5bd3f91d3d60dd1e649158e7fab908bda2bb30604f1ed8c08f37: Status 404 returned error can't find the container with id 4e2aecee09cf5bd3f91d3d60dd1e649158e7fab908bda2bb30604f1ed8c08f37 May 28 20:54:31.826376 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:31.826340 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" event={"ID":"f9e058d4-ea36-4595-8372-5f2839212a4b","Type":"ContainerStarted","Data":"82123a26eeb50eb8a8423ad4c9dbd1583b7fce863b483697dbedaf5fa655e476"} May 28 20:54:31.826376 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:31.826376 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" event={"ID":"f9e058d4-ea36-4595-8372-5f2839212a4b","Type":"ContainerStarted","Data":"4e2aecee09cf5bd3f91d3d60dd1e649158e7fab908bda2bb30604f1ed8c08f37"} May 28 20:54:31.841799 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:31.841749 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" podStartSLOduration=33.841733765 podStartE2EDuration="33.841733765s" podCreationTimestamp="2026-05-28 20:53:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 20:54:31.841268526 +0000 UTC m=+121.958662123" watchObservedRunningTime="2026-05-28 20:54:31.841733765 +0000 UTC m=+121.959127352" May 28 20:54:32.105135 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:32.105055 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:32.107625 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:32.107604 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:32.828470 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:32.828436 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:32.829673 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:32.829650 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6b5d5f9b5b-76hl9" May 28 20:54:34.198656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:34.198618 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw"] May 28 20:54:34.204630 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:34.204600 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw" May 28 20:54:34.213579 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:34.213550 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-x8978\"" May 28 20:54:34.214359 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:34.214334 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" May 28 20:54:34.215188 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:34.215136 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw"] May 28 20:54:34.299565 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:34.299533 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0bd2d4c1-7b03-4fa9-ada3-4403d1778e4a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw\" (UID: \"0bd2d4c1-7b03-4fa9-ada3-4403d1778e4a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw" May 28 20:54:34.399965 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:34.399935 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0bd2d4c1-7b03-4fa9-ada3-4403d1778e4a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw\" (UID: \"0bd2d4c1-7b03-4fa9-ada3-4403d1778e4a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw" May 28 20:54:34.402413 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:34.402381 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0bd2d4c1-7b03-4fa9-ada3-4403d1778e4a-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw\" (UID: \"0bd2d4c1-7b03-4fa9-ada3-4403d1778e4a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw" May 28 20:54:34.525473 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:34.525452 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw" May 28 20:54:34.645101 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:34.645072 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw"] May 28 20:54:34.648517 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:54:34.648491 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bd2d4c1_7b03_4fa9_ada3_4403d1778e4a.slice/crio-fa971a4e16dface29ec0c62f894713e27e711e93245ab0ee29a2b1ce8d94af0f WatchSource:0}: Error finding container fa971a4e16dface29ec0c62f894713e27e711e93245ab0ee29a2b1ce8d94af0f: Status 404 returned error can't find the container with id fa971a4e16dface29ec0c62f894713e27e711e93245ab0ee29a2b1ce8d94af0f May 28 20:54:34.833760 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:34.833680 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw" event={"ID":"0bd2d4c1-7b03-4fa9-ada3-4403d1778e4a","Type":"ContainerStarted","Data":"fa971a4e16dface29ec0c62f894713e27e711e93245ab0ee29a2b1ce8d94af0f"} May 28 20:54:35.837069 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:35.837001 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw" event={"ID":"0bd2d4c1-7b03-4fa9-ada3-4403d1778e4a","Type":"ContainerStarted","Data":"28e949e2af469dc789a2085642d8c3aeb8358be63e71ae18ff5ab61198e146e3"} May 28 20:54:35.837443 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:35.837202 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw" May 28 20:54:35.841821 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:35.841804 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw" May 28 20:54:35.851745 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:35.851699 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw" podStartSLOduration=0.949423679 podStartE2EDuration="1.851687799s" podCreationTimestamp="2026-05-28 20:54:34 +0000 UTC" firstStartedPulling="2026-05-28 20:54:34.650771838 +0000 UTC m=+124.768165402" lastFinishedPulling="2026-05-28 20:54:35.553035956 +0000 UTC m=+125.670429522" observedRunningTime="2026-05-28 20:54:35.850905463 +0000 UTC m=+125.968299047" watchObservedRunningTime="2026-05-28 20:54:35.851687799 +0000 UTC m=+125.969081384" May 28 20:54:36.264565 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.264531 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb"] May 28 20:54:36.267919 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.267896 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.270026 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.270006 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 28 20:54:36.270026 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.270021 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" May 28 20:54:36.270573 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.270555 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 28 20:54:36.270639 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.270582 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dbxkc\"" May 28 20:54:36.270639 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.270626 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 28 20:54:36.270794 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.270752 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" May 28 20:54:36.276071 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.276050 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb"] May 28 20:54:36.316001 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.315977 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxgx2\" (UniqueName: \"kubernetes.io/projected/09bff3df-cdf6-4388-85c9-400f4cfc9264-kube-api-access-pxgx2\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.316099 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.316028 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09bff3df-cdf6-4388-85c9-400f4cfc9264-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.316099 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.316051 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09bff3df-cdf6-4388-85c9-400f4cfc9264-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.316099 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.316080 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09bff3df-cdf6-4388-85c9-400f4cfc9264-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.417045 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.417019 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09bff3df-cdf6-4388-85c9-400f4cfc9264-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.417045 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.417045 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09bff3df-cdf6-4388-85c9-400f4cfc9264-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.417184 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.417064 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09bff3df-cdf6-4388-85c9-400f4cfc9264-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.417184 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.417145 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxgx2\" (UniqueName: \"kubernetes.io/projected/09bff3df-cdf6-4388-85c9-400f4cfc9264-kube-api-access-pxgx2\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.417259 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:36.417193 2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found May 28 20:54:36.417293 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:36.417270 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09bff3df-cdf6-4388-85c9-400f4cfc9264-prometheus-operator-tls podName:09bff3df-cdf6-4388-85c9-400f4cfc9264 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:36.917255405 +0000 UTC m=+127.034648973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/09bff3df-cdf6-4388-85c9-400f4cfc9264-prometheus-operator-tls") pod "prometheus-operator-7779bbdd78-6cbdb" (UID: "09bff3df-cdf6-4388-85c9-400f4cfc9264") : secret "prometheus-operator-tls" not found May 28 20:54:36.417668 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.417650 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09bff3df-cdf6-4388-85c9-400f4cfc9264-metrics-client-ca\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.419567 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.419542 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/09bff3df-cdf6-4388-85c9-400f4cfc9264-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.425342 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.425324 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxgx2\" (UniqueName: \"kubernetes.io/projected/09bff3df-cdf6-4388-85c9-400f4cfc9264-kube-api-access-pxgx2\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.920340 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.920305 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09bff3df-cdf6-4388-85c9-400f4cfc9264-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:36.922599 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:36.922579 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/09bff3df-cdf6-4388-85c9-400f4cfc9264-prometheus-operator-tls\") pod \"prometheus-operator-7779bbdd78-6cbdb\" (UID: \"09bff3df-cdf6-4388-85c9-400f4cfc9264\") " pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:37.177593 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:37.177514 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" May 28 20:54:37.293280 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:37.293251 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb"] May 28 20:54:37.296565 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:54:37.296537 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09bff3df_cdf6_4388_85c9_400f4cfc9264.slice/crio-36ef8a9b75911485c5958189f27e7d07e824eacee7fa77437e191ff68ed6dbb2 WatchSource:0}: Error finding container 36ef8a9b75911485c5958189f27e7d07e824eacee7fa77437e191ff68ed6dbb2: Status 404 returned error can't find the container with id 36ef8a9b75911485c5958189f27e7d07e824eacee7fa77437e191ff68ed6dbb2 May 28 20:54:37.842995 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:37.842951 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" event={"ID":"09bff3df-cdf6-4388-85c9-400f4cfc9264","Type":"ContainerStarted","Data":"36ef8a9b75911485c5958189f27e7d07e824eacee7fa77437e191ff68ed6dbb2"} May 28 20:54:39.850196 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:39.850151 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" event={"ID":"09bff3df-cdf6-4388-85c9-400f4cfc9264","Type":"ContainerStarted","Data":"de321726aa24b5099d7b51a0f02511897525905030d4d2620a4b8176b066d4b2"} May 28 20:54:39.850196 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:39.850196 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" event={"ID":"09bff3df-cdf6-4388-85c9-400f4cfc9264","Type":"ContainerStarted","Data":"6d89459c75ad0d7a53a7c35baa714222925ea198a1ab3876ee3dc43a4e5d3e22"} May 28 20:54:39.865797 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:39.865746 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7779bbdd78-6cbdb" podStartSLOduration=2.2758482620000002 podStartE2EDuration="3.865730275s" podCreationTimestamp="2026-05-28 20:54:36 +0000 UTC" firstStartedPulling="2026-05-28 20:54:37.298815464 +0000 UTC m=+127.416209027" lastFinishedPulling="2026-05-28 20:54:38.888697474 +0000 UTC m=+129.006091040" observedRunningTime="2026-05-28 20:54:39.864877548 +0000 UTC m=+129.982271133" watchObservedRunningTime="2026-05-28 20:54:39.865730275 +0000 UTC m=+129.983123862" May 28 20:54:40.244615 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:40.244590 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:54:40.246848 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:40.246824 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338d9053-7caa-48f3-bd43-6433cf054fc5-metrics-certs\") pod \"network-metrics-daemon-qqths\" (UID: \"338d9053-7caa-48f3-bd43-6433cf054fc5\") " pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:54:40.475734 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:40.475706 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kbvdl\"" May 28 20:54:40.484307 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:40.484290 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qqths" May 28 20:54:40.604306 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:40.604282 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qqths"] May 28 20:54:40.606774 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:54:40.606750 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338d9053_7caa_48f3_bd43_6433cf054fc5.slice/crio-4ebfecaefded4becc900774f7f52a10deecce2223852a8c18951b558bd87b435 WatchSource:0}: Error finding container 4ebfecaefded4becc900774f7f52a10deecce2223852a8c18951b558bd87b435: Status 404 returned error can't find the container with id 4ebfecaefded4becc900774f7f52a10deecce2223852a8c18951b558bd87b435 May 28 20:54:40.855983 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:40.855899 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qqths" event={"ID":"338d9053-7caa-48f3-bd43-6433cf054fc5","Type":"ContainerStarted","Data":"4ebfecaefded4becc900774f7f52a10deecce2223852a8c18951b558bd87b435"} May 28 20:54:41.637600 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.637565 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tnmmc"] May 28 20:54:41.640506 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.640486 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.642679 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.642581 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 28 20:54:41.642679 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.642652 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pxmvg\"" May 28 20:54:41.642854 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.642653 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 28 20:54:41.643028 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.643008 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 28 20:54:41.655232 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.655213 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-sys\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.655343 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.655251 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-textfile\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.655343 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.655293 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2t45\" (UniqueName: \"kubernetes.io/projected/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-kube-api-access-n2t45\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.655462 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.655371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-root\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.655462 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.655403 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-metrics-client-ca\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.655538 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.655491 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-tls\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.655584 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.655542 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.655584 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.655574 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-accelerators-collector-config\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.655672 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.655603 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-wtmp\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756456 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756432 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-metrics-client-ca\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756569 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756489 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-tls\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756569 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756519 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756569 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756549 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-accelerators-collector-config\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756731 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756580 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-wtmp\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756731 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756638 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-sys\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756731 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:41.756649 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found May 28 20:54:41.756731 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756672 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-textfile\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756731 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:41.756716 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-tls podName:c7dd10c3-795a-4a25-b7f4-6e5be725dc6f nodeName:}" failed. No retries permitted until 2026-05-28 20:54:42.256697634 +0000 UTC m=+132.374091219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-tls") pod "node-exporter-tnmmc" (UID: "c7dd10c3-795a-4a25-b7f4-6e5be725dc6f") : secret "node-exporter-tls" not found May 28 20:54:41.756986 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756729 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-sys\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756986 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756757 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n2t45\" (UniqueName: \"kubernetes.io/projected/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-kube-api-access-n2t45\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756986 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756759 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-wtmp\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756986 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756894 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-root\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756986 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756934 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-root\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.756986 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.756951 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-textfile\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.757258 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.757027 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-metrics-client-ca\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.757494 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.757474 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-accelerators-collector-config\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.759051 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.759021 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.764784 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.764725 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2t45\" (UniqueName: \"kubernetes.io/projected/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-kube-api-access-n2t45\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:41.861482 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:41.861440 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qqths" event={"ID":"338d9053-7caa-48f3-bd43-6433cf054fc5","Type":"ContainerStarted","Data":"9e31928a8cf295823477a3f22de88bf1067b22e0ea08166b76223c2780d22048"} May 28 20:54:42.259299 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.259267 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-tls\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:42.261512 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.261495 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7dd10c3-795a-4a25-b7f4-6e5be725dc6f-node-exporter-tls\") pod \"node-exporter-tnmmc\" (UID: \"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f\") " pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:42.552651 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.552568 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tnmmc" May 28 20:54:42.561822 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:54:42.561798 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7dd10c3_795a_4a25_b7f4_6e5be725dc6f.slice/crio-30224da1ee2a5ba4e3a4276b4e64e32e580710015ce6ce4ff04120f7cea7b000 WatchSource:0}: Error finding container 30224da1ee2a5ba4e3a4276b4e64e32e580710015ce6ce4ff04120f7cea7b000: Status 404 returned error can't find the container with id 30224da1ee2a5ba4e3a4276b4e64e32e580710015ce6ce4ff04120f7cea7b000 May 28 20:54:42.685551 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.685518 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 20:54:42.690924 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.690891 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.692983 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.692952 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" May 28 20:54:42.692983 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.692952 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" May 28 20:54:42.693160 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.693031 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" May 28 20:54:42.693160 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.693089 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" May 28 20:54:42.693160 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.693134 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" May 28 20:54:42.693160 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.693130 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jn6x2\"" May 28 20:54:42.693380 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.693257 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" May 28 20:54:42.693380 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.693296 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" May 28 20:54:42.693544 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.693529 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" May 28 20:54:42.698371 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.698353 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" May 28 20:54:42.700347 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.700327 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 20:54:42.761435 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761411 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761562 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761562 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761462 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-volume\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761562 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761509 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-out\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761657 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761565 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761657 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761593 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761657 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761625 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761747 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761685 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvs22\" (UniqueName: \"kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-kube-api-access-gvs22\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761747 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761718 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-web-config\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761807 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761763 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761807 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761784 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761875 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761811 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.761875 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.761830 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862324 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862252 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862324 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862281 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862324 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862309 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-volume\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862343 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-out\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862374 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:42.862402 2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862411 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862454 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:54:42.862471 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-main-tls podName:6cbb2ba6-3c4f-42bb-b04c-d33da370d34b nodeName:}" failed. No retries permitted until 2026-05-28 20:54:43.362448347 +0000 UTC m=+133.479841913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b") : secret "alertmanager-main-tls" not found May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862518 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gvs22\" (UniqueName: \"kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-kube-api-access-gvs22\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862548 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-web-config\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862614 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862640 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862661 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862677 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.862839 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.862714 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.863539 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.863111 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.863837 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.863810 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.865764 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.865736 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-web-config\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.865908 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.865886 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-out\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.866031 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.866010 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.866089 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.866050 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.866263 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.866235 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.866399 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.866374 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.866848 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.866829 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.867139 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.867117 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tnmmc" event={"ID":"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f","Type":"ContainerStarted","Data":"30224da1ee2a5ba4e3a4276b4e64e32e580710015ce6ce4ff04120f7cea7b000"} May 28 20:54:42.867745 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.867726 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-volume\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.868730 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.868710 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qqths" event={"ID":"338d9053-7caa-48f3-bd43-6433cf054fc5","Type":"ContainerStarted","Data":"bbe6bd3de81186e35397689016531fb9edd25a628f359a42e6885f1db525171c"} May 28 20:54:42.875051 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.875031 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvs22\" (UniqueName: \"kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-kube-api-access-gvs22\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:42.883064 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:42.883019 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qqths" podStartSLOduration=131.756269743 podStartE2EDuration="2m12.883005589s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:54:40.608659821 +0000 UTC m=+130.726053384" lastFinishedPulling="2026-05-28 20:54:41.735395664 +0000 UTC m=+131.852789230" observedRunningTime="2026-05-28 20:54:42.882538095 +0000 UTC m=+132.999931680" watchObservedRunningTime="2026-05-28 20:54:42.883005589 +0000 UTC m=+133.000399176" May 28 20:54:43.366445 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:43.366408 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:43.368856 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:43.368832 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:43.600573 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:43.600551 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 28 20:54:43.740273 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:43.740248 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 20:54:43.741772 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:54:43.741741 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cbb2ba6_3c4f_42bb_b04c_d33da370d34b.slice/crio-60510c255581186ca6cf3d2f1cd228a77b5910ac5409b6bcff18c0272a225876 WatchSource:0}: Error finding container 60510c255581186ca6cf3d2f1cd228a77b5910ac5409b6bcff18c0272a225876: Status 404 returned error can't find the container with id 60510c255581186ca6cf3d2f1cd228a77b5910ac5409b6bcff18c0272a225876 May 28 20:54:43.872100 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:43.872059 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerStarted","Data":"60510c255581186ca6cf3d2f1cd228a77b5910ac5409b6bcff18c0272a225876"} May 28 20:54:43.873644 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:43.873615 2563 generic.go:358] "Generic (PLEG): container finished" podID="c7dd10c3-795a-4a25-b7f4-6e5be725dc6f" containerID="ead3f01c4b7951ff7445999028e6a97160860bcaf9bc240fc3ac35555ea56879" exitCode=0 May 28 20:54:43.873808 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:43.873683 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tnmmc" event={"ID":"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f","Type":"ContainerDied","Data":"ead3f01c4b7951ff7445999028e6a97160860bcaf9bc240fc3ac35555ea56879"} May 28 20:54:44.878518 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:44.878482 2563 generic.go:358] "Generic (PLEG): container finished" podID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerID="99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f" exitCode=0 May 28 20:54:44.878942 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:44.878571 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerDied","Data":"99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f"} May 28 20:54:44.880543 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:44.880515 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tnmmc" event={"ID":"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f","Type":"ContainerStarted","Data":"b5de0ebd50e4fd67e0e79a810113e6697184166544369b36943086f6ea918501"} May 28 20:54:44.880631 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:44.880553 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tnmmc" event={"ID":"c7dd10c3-795a-4a25-b7f4-6e5be725dc6f","Type":"ContainerStarted","Data":"1e645f6c230a6eec4e62b98be46cc41feface72ba185a1323fee04cdf8a1378c"} May 28 20:54:44.919124 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:44.919078 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tnmmc" podStartSLOduration=2.9606835670000002 podStartE2EDuration="3.919060289s" podCreationTimestamp="2026-05-28 20:54:41 +0000 UTC" firstStartedPulling="2026-05-28 20:54:42.563453157 +0000 UTC m=+132.680846734" lastFinishedPulling="2026-05-28 20:54:43.521829894 +0000 UTC m=+133.639223456" observedRunningTime="2026-05-28 20:54:44.917669038 +0000 UTC m=+135.035062620" watchObservedRunningTime="2026-05-28 20:54:44.919060289 +0000 UTC m=+135.036453874" May 28 20:54:46.398674 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.398645 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5"] May 28 20:54:46.401782 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.401762 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5" May 28 20:54:46.404455 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.404406 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" May 28 20:54:46.404616 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.404594 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-j8dvs\"" May 28 20:54:46.411002 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.410968 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5"] May 28 20:54:46.494153 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.494122 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c2c83195-81e1-4d41-a670-5e3e9523f78f-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-qgkh5\" (UID: \"c2c83195-81e1-4d41-a670-5e3e9523f78f\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5" May 28 20:54:46.594802 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.594773 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c2c83195-81e1-4d41-a670-5e3e9523f78f-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-qgkh5\" (UID: \"c2c83195-81e1-4d41-a670-5e3e9523f78f\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5" May 28 20:54:46.597223 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.597201 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c2c83195-81e1-4d41-a670-5e3e9523f78f-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-qgkh5\" (UID: \"c2c83195-81e1-4d41-a670-5e3e9523f78f\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5" May 28 20:54:46.713469 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.713401 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5" May 28 20:54:46.839136 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.839107 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5"] May 28 20:54:46.842494 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:54:46.842461 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c83195_81e1_4d41_a670_5e3e9523f78f.slice/crio-a71b1595a9e7ae8e5b5252879c7e2c822cfb2ad3ee8913a4fdbd9699156a0f7e WatchSource:0}: Error finding container a71b1595a9e7ae8e5b5252879c7e2c822cfb2ad3ee8913a4fdbd9699156a0f7e: Status 404 returned error can't find the container with id a71b1595a9e7ae8e5b5252879c7e2c822cfb2ad3ee8913a4fdbd9699156a0f7e May 28 20:54:46.887412 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.887374 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5" event={"ID":"c2c83195-81e1-4d41-a670-5e3e9523f78f","Type":"ContainerStarted","Data":"a71b1595a9e7ae8e5b5252879c7e2c822cfb2ad3ee8913a4fdbd9699156a0f7e"} May 28 20:54:46.890483 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.890454 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerStarted","Data":"8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407"} May 28 20:54:46.890576 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.890491 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerStarted","Data":"c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856"} May 28 20:54:46.890576 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.890504 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerStarted","Data":"4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca"} May 28 20:54:46.890576 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.890517 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerStarted","Data":"77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4"} May 28 20:54:46.890576 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.890531 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerStarted","Data":"bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83"} May 28 20:54:46.948865 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.948838 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-dzhkc"] May 28 20:54:46.952263 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.952242 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-dzhkc" May 28 20:54:46.954247 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.954221 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 28 20:54:46.954247 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.954241 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 28 20:54:46.954429 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.954221 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9cdzm\"" May 28 20:54:46.961335 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.961267 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-dzhkc"] May 28 20:54:46.998943 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:46.998912 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzdhd\" (UniqueName: \"kubernetes.io/projected/1c261638-eb90-4ed1-8125-2210a314d8d8-kube-api-access-gzdhd\") pod \"downloads-57bd96b888-dzhkc\" (UID: \"1c261638-eb90-4ed1-8125-2210a314d8d8\") " pod="openshift-console/downloads-57bd96b888-dzhkc" May 28 20:54:47.099908 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.099885 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gzdhd\" (UniqueName: \"kubernetes.io/projected/1c261638-eb90-4ed1-8125-2210a314d8d8-kube-api-access-gzdhd\") pod \"downloads-57bd96b888-dzhkc\" (UID: \"1c261638-eb90-4ed1-8125-2210a314d8d8\") " pod="openshift-console/downloads-57bd96b888-dzhkc" May 28 20:54:47.107233 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.107205 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzdhd\" (UniqueName: \"kubernetes.io/projected/1c261638-eb90-4ed1-8125-2210a314d8d8-kube-api-access-gzdhd\") pod \"downloads-57bd96b888-dzhkc\" (UID: \"1c261638-eb90-4ed1-8125-2210a314d8d8\") " pod="openshift-console/downloads-57bd96b888-dzhkc" May 28 20:54:47.263845 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.263761 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-dzhkc" May 28 20:54:47.394716 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.394675 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-dzhkc"] May 28 20:54:47.397132 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:54:47.397104 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c261638_eb90_4ed1_8125_2210a314d8d8.slice/crio-073f19bf77b8ad70ad7834c457c62f109e4c0b3f45d9ffadc2336ae831319ccb WatchSource:0}: Error finding container 073f19bf77b8ad70ad7834c457c62f109e4c0b3f45d9ffadc2336ae831319ccb: Status 404 returned error can't find the container with id 073f19bf77b8ad70ad7834c457c62f109e4c0b3f45d9ffadc2336ae831319ccb May 28 20:54:47.853534 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.853503 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 20:54:47.858972 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.858939 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.863058 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.862951 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" May 28 20:54:47.863660 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.863640 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" May 28 20:54:47.863872 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.863854 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" May 28 20:54:47.864132 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.864114 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" May 28 20:54:47.864424 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.864400 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ab2jvkn6ueohg\"" May 28 20:54:47.865439 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.864619 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" May 28 20:54:47.865439 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.864837 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6sdtw\"" May 28 20:54:47.865439 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.864971 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" May 28 20:54:47.865439 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.865198 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" May 28 20:54:47.865439 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.865234 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" May 28 20:54:47.865743 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.865657 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" May 28 20:54:47.866301 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.866279 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" May 28 20:54:47.866398 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.866324 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" May 28 20:54:47.868034 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.867814 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" May 28 20:54:47.872587 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.872569 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" May 28 20:54:47.875083 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.875053 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.905767 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.905807 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.905833 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.905894 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.905930 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.905958 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.905982 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.906016 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.906109 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn7wm\" (UniqueName: \"kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-kube-api-access-nn7wm\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.906158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.906209 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.906235 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.906263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.906290 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.906313 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-web-config\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.908243 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.906341 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.909090 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.906366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config-out\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.909090 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.906388 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:47.909090 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.909064 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerStarted","Data":"1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017"} May 28 20:54:47.910947 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.910925 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-dzhkc" event={"ID":"1c261638-eb90-4ed1-8125-2210a314d8d8","Type":"ContainerStarted","Data":"073f19bf77b8ad70ad7834c457c62f109e4c0b3f45d9ffadc2336ae831319ccb"} May 28 20:54:47.934460 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:47.934399 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.58258235 podStartE2EDuration="5.934381504s" podCreationTimestamp="2026-05-28 20:54:42 +0000 UTC" firstStartedPulling="2026-05-28 20:54:43.743677465 +0000 UTC m=+133.861071027" lastFinishedPulling="2026-05-28 20:54:47.095476615 +0000 UTC m=+137.212870181" observedRunningTime="2026-05-28 20:54:47.932562682 +0000 UTC m=+138.049956272" watchObservedRunningTime="2026-05-28 20:54:47.934381504 +0000 UTC m=+138.051775091" May 28 20:54:48.007163 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007127 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007366 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007211 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007366 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007241 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007366 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007272 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007366 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007316 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007366 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007343 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-web-config\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007598 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007390 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007598 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007432 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config-out\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007598 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007455 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007598 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007484 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007598 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007511 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007740 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007887 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007851 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007887 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007855 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007920 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.007993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007973 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.008095 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.007999 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.008095 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.008058 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.009256 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.008156 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nn7wm\" (UniqueName: \"kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-kube-api-access-nn7wm\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.010682 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.009724 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.010682 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.010361 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.010913 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.010886 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.011816 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.011438 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.011988 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.011948 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.013919 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.013868 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.013919 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.013899 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.015450 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.015395 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.015646 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.015624 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.016269 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.016072 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config-out\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.016269 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.016222 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.016416 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.016293 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.017221 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.017197 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.017562 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.017545 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-web-config\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.017625 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.017600 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.018308 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.018288 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.020331 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.020278 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn7wm\" (UniqueName: \"kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-kube-api-access-nn7wm\") pod \"prometheus-k8s-0\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.174529 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.174396 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:54:48.655961 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.655934 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 20:54:48.657477 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:54:48.657447 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5f8c5e_e737_4e39_baf5_b0bc38698331.slice/crio-0dd38a76c11fff13352c0507670f83b777c2706c88fd1e25ab552e9dc3b2c98d WatchSource:0}: Error finding container 0dd38a76c11fff13352c0507670f83b777c2706c88fd1e25ab552e9dc3b2c98d: Status 404 returned error can't find the container with id 0dd38a76c11fff13352c0507670f83b777c2706c88fd1e25ab552e9dc3b2c98d May 28 20:54:48.916276 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.916208 2563 generic.go:358] "Generic (PLEG): container finished" podID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerID="30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982" exitCode=0 May 28 20:54:48.916690 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.916306 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerDied","Data":"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982"} May 28 20:54:48.916690 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.916349 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerStarted","Data":"0dd38a76c11fff13352c0507670f83b777c2706c88fd1e25ab552e9dc3b2c98d"} May 28 20:54:48.917972 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.917891 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5" event={"ID":"c2c83195-81e1-4d41-a670-5e3e9523f78f","Type":"ContainerStarted","Data":"a3412a2264b527be5e36f858c61b258430075e2ee528f530fcf63313262bb5f3"} May 28 20:54:48.918953 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.918927 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5" May 28 20:54:48.924303 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:48.924272 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5" May 28 20:54:50.483625 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:50.483568 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-qgkh5" podStartSLOduration=2.786294706 podStartE2EDuration="4.483549064s" podCreationTimestamp="2026-05-28 20:54:46 +0000 UTC" firstStartedPulling="2026-05-28 20:54:46.844685342 +0000 UTC m=+136.962078905" lastFinishedPulling="2026-05-28 20:54:48.541939697 +0000 UTC m=+138.659333263" observedRunningTime="2026-05-28 20:54:48.970216334 +0000 UTC m=+139.087609920" watchObservedRunningTime="2026-05-28 20:54:50.483549064 +0000 UTC m=+140.600942649" May 28 20:54:51.930857 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:51.930769 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerStarted","Data":"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc"} May 28 20:54:51.930857 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:51.930816 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerStarted","Data":"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af"} May 28 20:54:53.942255 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:53.942161 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerStarted","Data":"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508"} May 28 20:54:54.949351 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:54.949303 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerStarted","Data":"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604"} May 28 20:54:54.949351 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:54.949347 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerStarted","Data":"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488"} May 28 20:54:54.949818 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:54.949362 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerStarted","Data":"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c"} May 28 20:54:54.978550 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:54.978493 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.108639404 podStartE2EDuration="7.978478988s" podCreationTimestamp="2026-05-28 20:54:47 +0000 UTC" firstStartedPulling="2026-05-28 20:54:48.917608089 +0000 UTC m=+139.035001656" lastFinishedPulling="2026-05-28 20:54:53.787447663 +0000 UTC m=+143.904841240" observedRunningTime="2026-05-28 20:54:54.976506738 +0000 UTC m=+145.093900323" watchObservedRunningTime="2026-05-28 20:54:54.978478988 +0000 UTC m=+145.095872586" May 28 20:54:58.175214 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:54:58.175163 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:55:04.982423 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:04.982350 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-dzhkc" event={"ID":"1c261638-eb90-4ed1-8125-2210a314d8d8","Type":"ContainerStarted","Data":"c694688aa6658d10dc158f625b392066d1aa741c98283bfad802bca6eec6aa9c"} May 28 20:55:04.982887 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:04.982581 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-dzhkc" May 28 20:55:04.990036 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:04.990011 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-dzhkc" May 28 20:55:04.999737 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:04.999324 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-dzhkc" podStartSLOduration=1.665990154 podStartE2EDuration="18.999307763s" podCreationTimestamp="2026-05-28 20:54:46 +0000 UTC" firstStartedPulling="2026-05-28 20:54:47.399008093 +0000 UTC m=+137.516401661" lastFinishedPulling="2026-05-28 20:55:04.732325704 +0000 UTC m=+154.849719270" observedRunningTime="2026-05-28 20:55:04.996802451 +0000 UTC m=+155.114196036" watchObservedRunningTime="2026-05-28 20:55:04.999307763 +0000 UTC m=+155.116701349" May 28 20:55:09.308920 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:55:09.308868 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-h7jdr" podUID="c81ee9a4-8670-4a9f-8326-e30cf38d0ee9" May 28 20:55:09.331198 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:55:09.331141 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-s64pw" podUID="87bd9d8d-23ac-400a-8c2f-b368202b054e" May 28 20:55:09.997965 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:09.997926 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:55:09.998184 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:09.997926 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h7jdr" May 28 20:55:14.156120 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:14.156089 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:55:14.156623 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:14.156151 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:55:14.158461 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:14.158438 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87bd9d8d-23ac-400a-8c2f-b368202b054e-cert\") pod \"ingress-canary-s64pw\" (UID: \"87bd9d8d-23ac-400a-8c2f-b368202b054e\") " pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:55:14.158461 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:14.158450 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c81ee9a4-8670-4a9f-8326-e30cf38d0ee9-metrics-tls\") pod \"dns-default-h7jdr\" (UID: \"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9\") " pod="openshift-dns/dns-default-h7jdr" May 28 20:55:14.200502 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:14.200476 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gngfj\"" May 28 20:55:14.200766 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:14.200525 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6mfq4\"" May 28 20:55:14.209629 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:14.209606 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s64pw" May 28 20:55:14.209629 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:14.209621 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h7jdr" May 28 20:55:14.343056 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:14.342913 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h7jdr"] May 28 20:55:14.345708 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:55:14.345668 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81ee9a4_8670_4a9f_8326_e30cf38d0ee9.slice/crio-41ce561bec40bdd2f561488896a56f2bd915788be681af0ba97940b76af5d46e WatchSource:0}: Error finding container 41ce561bec40bdd2f561488896a56f2bd915788be681af0ba97940b76af5d46e: Status 404 returned error can't find the container with id 41ce561bec40bdd2f561488896a56f2bd915788be681af0ba97940b76af5d46e May 28 20:55:14.357959 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:14.357935 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s64pw"] May 28 20:55:14.360584 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:55:14.360560 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87bd9d8d_23ac_400a_8c2f_b368202b054e.slice/crio-a4c941d024b3065d61f0fff1da995ffd3b5638266177d650e83460b9f1491791 WatchSource:0}: Error finding container a4c941d024b3065d61f0fff1da995ffd3b5638266177d650e83460b9f1491791: Status 404 returned error can't find the container with id a4c941d024b3065d61f0fff1da995ffd3b5638266177d650e83460b9f1491791 May 28 20:55:15.015827 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:15.015771 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7jdr" event={"ID":"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9","Type":"ContainerStarted","Data":"41ce561bec40bdd2f561488896a56f2bd915788be681af0ba97940b76af5d46e"} May 28 20:55:15.017154 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:15.017116 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s64pw" event={"ID":"87bd9d8d-23ac-400a-8c2f-b368202b054e","Type":"ContainerStarted","Data":"a4c941d024b3065d61f0fff1da995ffd3b5638266177d650e83460b9f1491791"} May 28 20:55:17.025763 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:17.025712 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s64pw" event={"ID":"87bd9d8d-23ac-400a-8c2f-b368202b054e","Type":"ContainerStarted","Data":"708abec272dbd06cd6e2926649a6e72dc42356e3aab3f3d50ff6a7cdc6ad0984"} May 28 20:55:17.027457 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:17.027417 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7jdr" event={"ID":"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9","Type":"ContainerStarted","Data":"024de6dfd02f35f8bc5eb8caea6561b5d3e9b7891440d261f447570390b39022"} May 28 20:55:17.056781 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:17.056686 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s64pw" podStartSLOduration=128.58597418 podStartE2EDuration="2m11.056669394s" podCreationTimestamp="2026-05-28 20:53:06 +0000 UTC" firstStartedPulling="2026-05-28 20:55:14.362256821 +0000 UTC m=+164.479650384" lastFinishedPulling="2026-05-28 20:55:16.832952032 +0000 UTC m=+166.950345598" observedRunningTime="2026-05-28 20:55:17.055886083 +0000 UTC m=+167.173279669" watchObservedRunningTime="2026-05-28 20:55:17.056669394 +0000 UTC m=+167.174062981" May 28 20:55:18.031852 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:18.031816 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7jdr" event={"ID":"c81ee9a4-8670-4a9f-8326-e30cf38d0ee9","Type":"ContainerStarted","Data":"473eb4e36905f64720c858453a2b2c835fe312331cb0c7b77ff9135530688c66"} May 28 20:55:18.032313 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:18.032020 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h7jdr" May 28 20:55:18.051558 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:18.051516 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-h7jdr" podStartSLOduration=129.570662379 podStartE2EDuration="2m12.051505195s" podCreationTimestamp="2026-05-28 20:53:06 +0000 UTC" firstStartedPulling="2026-05-28 20:55:14.347559358 +0000 UTC m=+164.464952923" lastFinishedPulling="2026-05-28 20:55:16.82840217 +0000 UTC m=+166.945795739" observedRunningTime="2026-05-28 20:55:18.04994643 +0000 UTC m=+168.167340015" watchObservedRunningTime="2026-05-28 20:55:18.051505195 +0000 UTC m=+168.168898785" May 28 20:55:28.037055 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:28.037023 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h7jdr" May 28 20:55:28.060870 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:28.060845 2563 generic.go:358] "Generic (PLEG): container finished" podID="d4caaa6c-18e0-4e2b-a12c-1f596369b74a" containerID="167aaab911f160811060a6400adb926c6e288c49a7a6632805b1da327867a375" exitCode=0 May 28 20:55:28.061004 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:28.060916 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" event={"ID":"d4caaa6c-18e0-4e2b-a12c-1f596369b74a","Type":"ContainerDied","Data":"167aaab911f160811060a6400adb926c6e288c49a7a6632805b1da327867a375"} May 28 20:55:28.061272 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:28.061245 2563 scope.go:117] "RemoveContainer" containerID="167aaab911f160811060a6400adb926c6e288c49a7a6632805b1da327867a375" May 28 20:55:28.062351 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:28.062329 2563 generic.go:358] "Generic (PLEG): container finished" podID="ec7c2b71-2588-4c6a-93d0-be7df0291377" containerID="8054b3f8d842fe2114cb2fa655d1b27bd18fa26f131d47a6ffd6215636f085ab" exitCode=0 May 28 20:55:28.062439 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:28.062382 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" event={"ID":"ec7c2b71-2588-4c6a-93d0-be7df0291377","Type":"ContainerDied","Data":"8054b3f8d842fe2114cb2fa655d1b27bd18fa26f131d47a6ffd6215636f085ab"} May 28 20:55:28.062640 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:28.062627 2563 scope.go:117] "RemoveContainer" containerID="8054b3f8d842fe2114cb2fa655d1b27bd18fa26f131d47a6ffd6215636f085ab" May 28 20:55:29.066457 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:29.066422 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-ppdw9" event={"ID":"ec7c2b71-2588-4c6a-93d0-be7df0291377","Type":"ContainerStarted","Data":"41164046f4db983a71c01260025d5bbcc82f49c0ad195d3ee9e5f57bf2c1b369"} May 28 20:55:29.067992 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:29.067969 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-hrsg6" event={"ID":"d4caaa6c-18e0-4e2b-a12c-1f596369b74a","Type":"ContainerStarted","Data":"4a30b2ba53dc6ded82a5a7cc2f67584fcf197ba2a45f338b100973376098480d"} May 28 20:55:48.175128 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:48.175095 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:55:48.191276 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:48.191251 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:55:49.143389 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:55:49.143362 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:01.905480 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:01.905440 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 20:56:01.906052 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:01.905862 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="alertmanager" containerID="cri-o://bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83" gracePeriod=120 May 28 20:56:01.906052 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:01.905925 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy-metric" containerID="cri-o://8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407" gracePeriod=120 May 28 20:56:01.906052 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:01.905987 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="prom-label-proxy" containerID="cri-o://1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017" gracePeriod=120 May 28 20:56:01.906052 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:01.905962 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy-web" containerID="cri-o://4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca" gracePeriod=120 May 28 20:56:01.906052 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:01.906039 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy" containerID="cri-o://c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856" gracePeriod=120 May 28 20:56:01.906332 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:01.906017 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="config-reloader" containerID="cri-o://77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4" gracePeriod=120 May 28 20:56:02.167494 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:02.167409 2563 generic.go:358] "Generic (PLEG): container finished" podID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerID="1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017" exitCode=0 May 28 20:56:02.167494 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:02.167437 2563 generic.go:358] "Generic (PLEG): container finished" podID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerID="c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856" exitCode=0 May 28 20:56:02.167494 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:02.167445 2563 generic.go:358] "Generic (PLEG): container finished" podID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerID="77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4" exitCode=0 May 28 20:56:02.167494 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:02.167451 2563 generic.go:358] "Generic (PLEG): container finished" podID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerID="bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83" exitCode=0 May 28 20:56:02.167494 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:02.167479 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerDied","Data":"1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017"} May 28 20:56:02.167755 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:02.167513 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerDied","Data":"c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856"} May 28 20:56:02.167755 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:02.167524 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerDied","Data":"77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4"} May 28 20:56:02.167755 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:02.167533 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerDied","Data":"bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83"} May 28 20:56:03.143613 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.143585 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.173798 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.173762 2563 generic.go:358] "Generic (PLEG): container finished" podID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerID="8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407" exitCode=0 May 28 20:56:03.173798 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.173796 2563 generic.go:358] "Generic (PLEG): container finished" podID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerID="4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca" exitCode=0 May 28 20:56:03.174025 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.173822 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerDied","Data":"8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407"} May 28 20:56:03.174025 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.173857 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerDied","Data":"4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca"} May 28 20:56:03.174025 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.173872 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b","Type":"ContainerDied","Data":"60510c255581186ca6cf3d2f1cd228a77b5910ac5409b6bcff18c0272a225876"} May 28 20:56:03.174025 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.173891 2563 scope.go:117] "RemoveContainer" containerID="1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017" May 28 20:56:03.174025 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.173890 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.184595 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.184568 2563 scope.go:117] "RemoveContainer" containerID="8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407" May 28 20:56:03.194284 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.194183 2563 scope.go:117] "RemoveContainer" containerID="c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856" May 28 20:56:03.201436 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.201416 2563 scope.go:117] "RemoveContainer" containerID="4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca" May 28 20:56:03.208599 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.208579 2563 scope.go:117] "RemoveContainer" containerID="77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4" May 28 20:56:03.215270 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.215250 2563 scope.go:117] "RemoveContainer" containerID="bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83" May 28 20:56:03.219322 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219289 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-volume\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.219410 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219339 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-trusted-ca-bundle\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.219410 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219388 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-cluster-tls-config\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.219515 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219424 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.219572 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219532 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-tls-assets\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.219625 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219593 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-main-db\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.219677 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219633 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.219677 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219666 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-metrics-client-ca\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.219773 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219693 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-main-tls\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.219773 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219724 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-web-config\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.219773 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219759 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-out\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.219920 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219795 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 20:56:03.219920 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219802 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-web\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.220028 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219948 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 20:56:03.220028 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.219963 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvs22\" (UniqueName: \"kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-kube-api-access-gvs22\") pod \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\" (UID: \"6cbb2ba6-3c4f-42bb-b04c-d33da370d34b\") " May 28 20:56:03.220244 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.220223 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.220334 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.220250 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-alertmanager-main-db\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.221047 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.221003 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 20:56:03.224053 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.223988 2563 scope.go:117] "RemoveContainer" containerID="99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f" May 28 20:56:03.224202 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.224155 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-kube-api-access-gvs22" (OuterVolumeSpecName: "kube-api-access-gvs22") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "kube-api-access-gvs22". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 20:56:03.224409 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.224285 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:03.224659 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.224555 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:03.225415 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.225388 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:03.225811 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.225736 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:03.226070 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.226045 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-volume" (OuterVolumeSpecName: "config-volume") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:03.226163 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.226087 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 20:56:03.226304 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.226283 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-out" (OuterVolumeSpecName: "config-out") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 20:56:03.229328 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.229214 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:03.235121 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.235094 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-web-config" (OuterVolumeSpecName: "web-config") pod "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" (UID: "6cbb2ba6-3c4f-42bb-b04c-d33da370d34b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:03.236975 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.236957 2563 scope.go:117] "RemoveContainer" containerID="1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017" May 28 20:56:03.237306 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:03.237282 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017\": container with ID starting with 1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017 not found: ID does not exist" containerID="1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017" May 28 20:56:03.237365 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.237316 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017"} err="failed to get container status \"1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017\": rpc error: code = NotFound desc = could not find container \"1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017\": container with ID starting with 1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017 not found: ID does not exist" May 28 20:56:03.237365 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.237350 2563 scope.go:117] "RemoveContainer" containerID="8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407" May 28 20:56:03.237610 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:03.237582 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407\": container with ID starting with 8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407 not found: ID does not exist" containerID="8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407" May 28 20:56:03.237653 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.237608 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407"} err="failed to get container status \"8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407\": rpc error: code = NotFound desc = could not find container \"8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407\": container with ID starting with 8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407 not found: ID does not exist" May 28 20:56:03.237653 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.237631 2563 scope.go:117] "RemoveContainer" containerID="c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856" May 28 20:56:03.237878 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:03.237858 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856\": container with ID starting with c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856 not found: ID does not exist" containerID="c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856" May 28 20:56:03.237962 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.237879 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856"} err="failed to get container status \"c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856\": rpc error: code = NotFound desc = could not find container \"c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856\": container with ID starting with c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856 not found: ID does not exist" May 28 20:56:03.237962 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.237904 2563 scope.go:117] "RemoveContainer" containerID="4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca" May 28 20:56:03.238116 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:03.238103 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca\": container with ID starting with 4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca not found: ID does not exist" containerID="4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca" May 28 20:56:03.238154 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.238118 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca"} err="failed to get container status \"4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca\": rpc error: code = NotFound desc = could not find container \"4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca\": container with ID starting with 4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca not found: ID does not exist" May 28 20:56:03.238154 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.238131 2563 scope.go:117] "RemoveContainer" containerID="77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4" May 28 20:56:03.238353 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:03.238339 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4\": container with ID starting with 77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4 not found: ID does not exist" containerID="77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4" May 28 20:56:03.238402 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.238356 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4"} err="failed to get container status \"77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4\": rpc error: code = NotFound desc = could not find container \"77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4\": container with ID starting with 77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4 not found: ID does not exist" May 28 20:56:03.238402 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.238367 2563 scope.go:117] "RemoveContainer" containerID="bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83" May 28 20:56:03.238563 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:03.238547 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83\": container with ID starting with bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83 not found: ID does not exist" containerID="bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83" May 28 20:56:03.238602 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.238568 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83"} err="failed to get container status \"bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83\": rpc error: code = NotFound desc = could not find container \"bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83\": container with ID starting with bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83 not found: ID does not exist" May 28 20:56:03.238602 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.238582 2563 scope.go:117] "RemoveContainer" containerID="99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f" May 28 20:56:03.238780 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:03.238764 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f\": container with ID starting with 99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f not found: ID does not exist" containerID="99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f" May 28 20:56:03.238825 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.238782 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f"} err="failed to get container status \"99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f\": rpc error: code = NotFound desc = could not find container \"99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f\": container with ID starting with 99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f not found: ID does not exist" May 28 20:56:03.238825 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.238793 2563 scope.go:117] "RemoveContainer" containerID="1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017" May 28 20:56:03.238978 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.238954 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017"} err="failed to get container status \"1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017\": rpc error: code = NotFound desc = could not find container \"1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017\": container with ID starting with 1168cd9b3f62efbdd8bc4d07858a0fee7b4423172be7f844f5a9bb28769f7017 not found: ID does not exist" May 28 20:56:03.239020 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.238979 2563 scope.go:117] "RemoveContainer" containerID="8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407" May 28 20:56:03.239136 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.239122 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407"} err="failed to get container status \"8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407\": rpc error: code = NotFound desc = could not find container \"8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407\": container with ID starting with 8f249b343c37b22680481b056ac5c4c6d962291e6b9e90dbbe953eca981c9407 not found: ID does not exist" May 28 20:56:03.239222 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.239144 2563 scope.go:117] "RemoveContainer" containerID="c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856" May 28 20:56:03.239345 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.239325 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856"} err="failed to get container status \"c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856\": rpc error: code = NotFound desc = could not find container \"c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856\": container with ID starting with c4c67a80e49aa7b37ae11d5fb5842b7821e82ef9a2504da72d98a47b9e215856 not found: ID does not exist" May 28 20:56:03.239403 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.239345 2563 scope.go:117] "RemoveContainer" containerID="4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca" May 28 20:56:03.239525 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.239508 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca"} err="failed to get container status \"4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca\": rpc error: code = NotFound desc = could not find container \"4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca\": container with ID starting with 4ad00af48be4b029e674e292162a82a60a9df68cf607965f701480413eec28ca not found: ID does not exist" May 28 20:56:03.239569 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.239525 2563 scope.go:117] "RemoveContainer" containerID="77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4" May 28 20:56:03.239699 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.239682 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4"} err="failed to get container status \"77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4\": rpc error: code = NotFound desc = could not find container \"77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4\": container with ID starting with 77ad69975e1c45369b2361ffa0831cb4ad25501d2e06b7aa48ad742ff43682b4 not found: ID does not exist" May 28 20:56:03.239741 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.239700 2563 scope.go:117] "RemoveContainer" containerID="bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83" May 28 20:56:03.239884 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.239869 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83"} err="failed to get container status \"bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83\": rpc error: code = NotFound desc = could not find container \"bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83\": container with ID starting with bbe6a0c39490b01a8d278e2bfca748414b8b65743bc57a217019e322dfe51c83 not found: ID does not exist" May 28 20:56:03.239933 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.239884 2563 scope.go:117] "RemoveContainer" containerID="99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f" May 28 20:56:03.240051 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.240035 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f"} err="failed to get container status \"99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f\": rpc error: code = NotFound desc = could not find container \"99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f\": container with ID starting with 99e53dbd715efb9a2c4c4be4797a94ed56c31299bdba524e19739fec55ea697f not found: ID does not exist" May 28 20:56:03.320742 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.320706 2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-cluster-tls-config\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.320742 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.320740 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.320906 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.320778 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-tls-assets\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.320906 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.320788 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.320906 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.320798 2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-metrics-client-ca\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.320906 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.320808 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-main-tls\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.320906 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.320819 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-web-config\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.320906 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.320829 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-out\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.320906 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.320837 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.320906 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.320846 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvs22\" (UniqueName: \"kubernetes.io/projected/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-kube-api-access-gvs22\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.320906 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.320854 2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b-config-volume\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:03.499374 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.499345 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 20:56:03.504190 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.504151 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 20:56:03.527656 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.527625 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 20:56:03.527962 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.527948 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="alertmanager" May 28 20:56:03.528005 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.527967 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="alertmanager" May 28 20:56:03.528005 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.527986 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy" May 28 20:56:03.528005 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.527993 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy" May 28 20:56:03.528005 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.527999 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="config-reloader" May 28 20:56:03.528005 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528005 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="config-reloader" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528011 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy-metric" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528016 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy-metric" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528025 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy-web" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528030 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy-web" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528042 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="init-config-reloader" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528050 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="init-config-reloader" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528060 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="prom-label-proxy" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528065 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="prom-label-proxy" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528109 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy-metric" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528119 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="config-reloader" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528128 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="alertmanager" May 28 20:56:03.528144 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528139 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="prom-label-proxy" May 28 20:56:03.528485 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528151 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy" May 28 20:56:03.528485 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.528160 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" containerName="kube-rbac-proxy-web" May 28 20:56:03.531298 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.531275 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.533675 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.533531 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" May 28 20:56:03.533675 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.533542 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" May 28 20:56:03.533675 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.533554 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jn6x2\"" May 28 20:56:03.533675 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.533566 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" May 28 20:56:03.533675 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.533537 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" May 28 20:56:03.533675 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.533580 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" May 28 20:56:03.533675 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.533643 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" May 28 20:56:03.534034 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.533952 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" May 28 20:56:03.534034 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.533958 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" May 28 20:56:03.538233 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.538215 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" May 28 20:56:03.543186 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.543152 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 20:56:03.622536 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622502 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-config-volume\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.622703 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622541 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c0c283-4236-4cb1-a514-4ff3f334c8ef-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.622703 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622561 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8qd2\" (UniqueName: \"kubernetes.io/projected/28c0c283-4236-4cb1-a514-4ff3f334c8ef-kube-api-access-d8qd2\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.622703 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622623 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28c0c283-4236-4cb1-a514-4ff3f334c8ef-config-out\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.622703 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622660 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.622833 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622704 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-web-config\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.622833 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622739 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.622833 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622757 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c0c283-4236-4cb1-a514-4ff3f334c8ef-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.622833 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622779 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.622833 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622803 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28c0c283-4236-4cb1-a514-4ff3f334c8ef-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.622833 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622823 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.623000 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622845 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28c0c283-4236-4cb1-a514-4ff3f334c8ef-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.623000 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.622866 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723473 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723444 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-web-config\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723574 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723478 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723574 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723496 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c0c283-4236-4cb1-a514-4ff3f334c8ef-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723574 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723518 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723574 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723544 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28c0c283-4236-4cb1-a514-4ff3f334c8ef-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723574 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723570 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723803 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723603 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28c0c283-4236-4cb1-a514-4ff3f334c8ef-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723803 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723627 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723803 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723672 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-config-volume\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723803 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723705 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c0c283-4236-4cb1-a514-4ff3f334c8ef-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723803 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723726 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d8qd2\" (UniqueName: \"kubernetes.io/projected/28c0c283-4236-4cb1-a514-4ff3f334c8ef-kube-api-access-d8qd2\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723803 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723752 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28c0c283-4236-4cb1-a514-4ff3f334c8ef-config-out\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.723803 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.723779 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.724135 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.724031 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/28c0c283-4236-4cb1-a514-4ff3f334c8ef-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.724433 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.724409 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28c0c283-4236-4cb1-a514-4ff3f334c8ef-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.725033 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.725005 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28c0c283-4236-4cb1-a514-4ff3f334c8ef-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.726641 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.726610 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-web-config\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.726734 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.726646 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.726734 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.726689 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.726849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.726761 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.726849 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.726759 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.727039 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.727020 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28c0c283-4236-4cb1-a514-4ff3f334c8ef-tls-assets\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.727248 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.727229 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.727463 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.727448 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/28c0c283-4236-4cb1-a514-4ff3f334c8ef-config-volume\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.728344 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.728328 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28c0c283-4236-4cb1-a514-4ff3f334c8ef-config-out\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.735063 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.735020 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8qd2\" (UniqueName: \"kubernetes.io/projected/28c0c283-4236-4cb1-a514-4ff3f334c8ef-kube-api-access-d8qd2\") pod \"alertmanager-main-0\" (UID: \"28c0c283-4236-4cb1-a514-4ff3f334c8ef\") " pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.841938 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.841904 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" May 28 20:56:03.976947 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:03.976884 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] May 28 20:56:03.980327 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:56:03.980301 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c0c283_4236_4cb1_a514_4ff3f334c8ef.slice/crio-a14a0c8c54b9b190908918c527965f9c732955352a0e8aa4d8ee6e7db1070c2d WatchSource:0}: Error finding container a14a0c8c54b9b190908918c527965f9c732955352a0e8aa4d8ee6e7db1070c2d: Status 404 returned error can't find the container with id a14a0c8c54b9b190908918c527965f9c732955352a0e8aa4d8ee6e7db1070c2d May 28 20:56:04.178234 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:04.178180 2563 generic.go:358] "Generic (PLEG): container finished" podID="28c0c283-4236-4cb1-a514-4ff3f334c8ef" containerID="79303a5a38bdf1bc05044ab002ef57c71998b7e1cea95c2adbe1d1c97f175d0d" exitCode=0 May 28 20:56:04.178721 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:04.178287 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c0c283-4236-4cb1-a514-4ff3f334c8ef","Type":"ContainerDied","Data":"79303a5a38bdf1bc05044ab002ef57c71998b7e1cea95c2adbe1d1c97f175d0d"} May 28 20:56:04.178721 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:04.178329 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c0c283-4236-4cb1-a514-4ff3f334c8ef","Type":"ContainerStarted","Data":"a14a0c8c54b9b190908918c527965f9c732955352a0e8aa4d8ee6e7db1070c2d"} May 28 20:56:04.467915 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:04.467886 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cbb2ba6-3c4f-42bb-b04c-d33da370d34b" path="/var/lib/kubelet/pods/6cbb2ba6-3c4f-42bb-b04c-d33da370d34b/volumes" May 28 20:56:05.185377 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:05.185347 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c0c283-4236-4cb1-a514-4ff3f334c8ef","Type":"ContainerStarted","Data":"6319d2cf27b4240d42e25356da864cc6bff0ac4280ba78e05ec917ec79061d4f"} May 28 20:56:05.185377 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:05.185380 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c0c283-4236-4cb1-a514-4ff3f334c8ef","Type":"ContainerStarted","Data":"e79fca932a93b35caaf35b4b3449de70df5d80af08d93d9242f55a1ac8466f5c"} May 28 20:56:05.185750 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:05.185390 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c0c283-4236-4cb1-a514-4ff3f334c8ef","Type":"ContainerStarted","Data":"9affec294463fc0eaff482d85ab378c7ff33b5ce13ff54a7f24dca0f9f925b10"} May 28 20:56:05.185750 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:05.185400 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c0c283-4236-4cb1-a514-4ff3f334c8ef","Type":"ContainerStarted","Data":"1a99c6385950a790e049f03f43b6a42399f6d7f9e7cc69a2ec22cb8e0790ade0"} May 28 20:56:05.185750 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:05.185409 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c0c283-4236-4cb1-a514-4ff3f334c8ef","Type":"ContainerStarted","Data":"529bfc434a863bae80141c7104c189bf66f9bd3190c2c1a46ee8d2cf1f64a0b2"} May 28 20:56:05.185750 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:05.185416 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"28c0c283-4236-4cb1-a514-4ff3f334c8ef","Type":"ContainerStarted","Data":"269bc7c9839f16461c094255927d9c1b9749e64dfcf0944e67aabe8dda44e17a"} May 28 20:56:05.209327 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:05.209247 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.209233166 podStartE2EDuration="2.209233166s" podCreationTimestamp="2026-05-28 20:56:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 20:56:05.208416631 +0000 UTC m=+215.325810228" watchObservedRunningTime="2026-05-28 20:56:05.209233166 +0000 UTC m=+215.326626750" May 28 20:56:06.229448 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.229411 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 20:56:06.230522 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.230486 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="prometheus" containerID="cri-o://f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af" gracePeriod=600 May 28 20:56:06.230713 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.230532 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="config-reloader" containerID="cri-o://c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc" gracePeriod=600 May 28 20:56:06.230842 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.230540 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy-thanos" containerID="cri-o://4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488" gracePeriod=600 May 28 20:56:06.230903 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.230558 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy-web" containerID="cri-o://99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604" gracePeriod=600 May 28 20:56:06.230957 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.230540 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="thanos-sidecar" containerID="cri-o://2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508" gracePeriod=600 May 28 20:56:06.230957 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.230573 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy" containerID="cri-o://9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c" gracePeriod=600 May 28 20:56:06.494530 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.494504 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.548123 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-tls\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.548775 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-kube-rbac-proxy\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.548812 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-metrics-client-ca\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.548853 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.548889 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-tls-assets\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.548915 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-serving-certs-ca-bundle\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.548940 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.548974 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-db\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.549006 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nn7wm\" (UniqueName: \"kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-kube-api-access-nn7wm\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.549048 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-kubelet-serving-ca-bundle\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.549077 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-web-config\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.549111 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-rulefiles-0\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.549148 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.549195 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-metrics-client-certs\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.549226 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-thanos-prometheus-http-client-file\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.549266 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config-out\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.549291 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-grpc-tls\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.549694 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.549332 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-trusted-ca-bundle\") pod \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\" (UID: \"bb5f8c5e-e737-4e39-baf5-b0bc38698331\") " May 28 20:56:06.550736 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.549919 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 20:56:06.550736 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.550522 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 20:56:06.551501 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.551417 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 20:56:06.552458 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.552180 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 20:56:06.553479 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.552704 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 20:56:06.553588 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.553553 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config-out" (OuterVolumeSpecName: "config-out") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 20:56:06.554037 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.553995 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 20:56:06.555603 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.555574 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:06.556270 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.556233 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:06.556611 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.556494 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config" (OuterVolumeSpecName: "config") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:06.556611 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.556536 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:06.556611 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.556571 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:06.556853 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.556634 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-kube-api-access-nn7wm" (OuterVolumeSpecName: "kube-api-access-nn7wm") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "kube-api-access-nn7wm". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 20:56:06.556853 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.556683 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:06.556853 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.556695 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:06.557132 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.557113 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 20:56:06.558426 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.558391 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:06.568936 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.568908 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-web-config" (OuterVolumeSpecName: "web-config") pod "bb5f8c5e-e737-4e39-baf5-b0bc38698331" (UID: "bb5f8c5e-e737-4e39-baf5-b0bc38698331"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:56:06.650639 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650607 2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650639 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650632 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-tls-assets\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650639 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650643 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650653 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650664 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-db\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650673 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn7wm\" (UniqueName: \"kubernetes.io/projected/bb5f8c5e-e737-4e39-baf5-b0bc38698331-kube-api-access-nn7wm\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650684 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650695 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-web-config\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650704 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650714 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650723 2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-metrics-client-certs\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650731 2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650740 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb5f8c5e-e737-4e39-baf5-b0bc38698331-config-out\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650748 2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-grpc-tls\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650757 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650767 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650776 2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb5f8c5e-e737-4e39-baf5-b0bc38698331-secret-kube-rbac-proxy\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:06.650860 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:06.650784 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb5f8c5e-e737-4e39-baf5-b0bc38698331-configmap-metrics-client-ca\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 20:56:07.194217 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194180 2563 generic.go:358] "Generic (PLEG): container finished" podID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerID="4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488" exitCode=0 May 28 20:56:07.194217 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194209 2563 generic.go:358] "Generic (PLEG): container finished" podID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerID="9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c" exitCode=0 May 28 20:56:07.194217 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194216 2563 generic.go:358] "Generic (PLEG): container finished" podID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerID="99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604" exitCode=0 May 28 20:56:07.194217 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194222 2563 generic.go:358] "Generic (PLEG): container finished" podID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerID="2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508" exitCode=0 May 28 20:56:07.194217 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194227 2563 generic.go:358] "Generic (PLEG): container finished" podID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerID="c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc" exitCode=0 May 28 20:56:07.194646 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194232 2563 generic.go:358] "Generic (PLEG): container finished" podID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerID="f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af" exitCode=0 May 28 20:56:07.194646 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194258 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerDied","Data":"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488"} May 28 20:56:07.194646 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194280 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.194646 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194302 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerDied","Data":"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c"} May 28 20:56:07.194646 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194313 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerDied","Data":"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604"} May 28 20:56:07.194646 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194322 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerDied","Data":"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508"} May 28 20:56:07.194646 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194331 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerDied","Data":"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc"} May 28 20:56:07.194646 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194341 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerDied","Data":"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af"} May 28 20:56:07.194646 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194349 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb5f8c5e-e737-4e39-baf5-b0bc38698331","Type":"ContainerDied","Data":"0dd38a76c11fff13352c0507670f83b777c2706c88fd1e25ab552e9dc3b2c98d"} May 28 20:56:07.194646 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.194364 2563 scope.go:117] "RemoveContainer" containerID="4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488" May 28 20:56:07.201919 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.201883 2563 scope.go:117] "RemoveContainer" containerID="9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c" May 28 20:56:07.208616 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.208600 2563 scope.go:117] "RemoveContainer" containerID="99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604" May 28 20:56:07.214944 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.214930 2563 scope.go:117] "RemoveContainer" containerID="2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508" May 28 20:56:07.217621 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.217582 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 20:56:07.222349 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.222330 2563 scope.go:117] "RemoveContainer" containerID="c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc" May 28 20:56:07.224470 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.224446 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 20:56:07.229016 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.228999 2563 scope.go:117] "RemoveContainer" containerID="f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af" May 28 20:56:07.235544 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.235415 2563 scope.go:117] "RemoveContainer" containerID="30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982" May 28 20:56:07.254290 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254270 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 20:56:07.254584 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254570 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="thanos-sidecar" May 28 20:56:07.254647 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254587 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="thanos-sidecar" May 28 20:56:07.254647 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254601 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="prometheus" May 28 20:56:07.254647 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254606 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="prometheus" May 28 20:56:07.254647 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254614 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy-thanos" May 28 20:56:07.254647 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254620 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy-thanos" May 28 20:56:07.254647 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254633 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="init-config-reloader" May 28 20:56:07.254647 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254638 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="init-config-reloader" May 28 20:56:07.254647 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254645 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy" May 28 20:56:07.254647 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254650 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy" May 28 20:56:07.254923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254657 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy-web" May 28 20:56:07.254923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254662 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy-web" May 28 20:56:07.254923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254668 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="config-reloader" May 28 20:56:07.254923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254673 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="config-reloader" May 28 20:56:07.254923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254715 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy-thanos" May 28 20:56:07.254923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254724 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy-web" May 28 20:56:07.254923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254731 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="config-reloader" May 28 20:56:07.254923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254737 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="prometheus" May 28 20:56:07.254923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254743 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="thanos-sidecar" May 28 20:56:07.254923 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.254748 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" containerName="kube-rbac-proxy" May 28 20:56:07.256565 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.256547 2563 scope.go:117] "RemoveContainer" containerID="4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488" May 28 20:56:07.256860 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:07.256836 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": container with ID starting with 4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488 not found: ID does not exist" containerID="4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488" May 28 20:56:07.256938 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.256867 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488"} err="failed to get container status \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": rpc error: code = NotFound desc = could not find container \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": container with ID starting with 4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488 not found: ID does not exist" May 28 20:56:07.256938 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.256886 2563 scope.go:117] "RemoveContainer" containerID="9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c" May 28 20:56:07.257211 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:07.257187 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": container with ID starting with 9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c not found: ID does not exist" containerID="9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c" May 28 20:56:07.257323 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.257220 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c"} err="failed to get container status \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": rpc error: code = NotFound desc = could not find container \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": container with ID starting with 9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c not found: ID does not exist" May 28 20:56:07.257323 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.257239 2563 scope.go:117] "RemoveContainer" containerID="99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604" May 28 20:56:07.257683 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:07.257659 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": container with ID starting with 99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604 not found: ID does not exist" containerID="99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604" May 28 20:56:07.259658 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.257688 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604"} err="failed to get container status \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": rpc error: code = NotFound desc = could not find container \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": container with ID starting with 99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604 not found: ID does not exist" May 28 20:56:07.259658 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.257718 2563 scope.go:117] "RemoveContainer" containerID="2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508" May 28 20:56:07.259658 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:07.258019 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": container with ID starting with 2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508 not found: ID does not exist" containerID="2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508" May 28 20:56:07.259658 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.258044 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508"} err="failed to get container status \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": rpc error: code = NotFound desc = could not find container \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": container with ID starting with 2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508 not found: ID does not exist" May 28 20:56:07.259658 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.258071 2563 scope.go:117] "RemoveContainer" containerID="c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc" May 28 20:56:07.259658 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.259361 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.259946 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:07.259825 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": container with ID starting with c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc not found: ID does not exist" containerID="c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc" May 28 20:56:07.259946 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.259855 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc"} err="failed to get container status \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": rpc error: code = NotFound desc = could not find container \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": container with ID starting with c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc not found: ID does not exist" May 28 20:56:07.259946 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.259877 2563 scope.go:117] "RemoveContainer" containerID="f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af" May 28 20:56:07.260355 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:07.260298 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": container with ID starting with f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af not found: ID does not exist" containerID="f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af" May 28 20:56:07.260355 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.260331 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af"} err="failed to get container status \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": rpc error: code = NotFound desc = could not find container \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": container with ID starting with f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af not found: ID does not exist" May 28 20:56:07.260355 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.260354 2563 scope.go:117] "RemoveContainer" containerID="30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982" May 28 20:56:07.260616 ip-10-0-140-94 kubenswrapper[2563]: E0528 20:56:07.260589 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": container with ID starting with 30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982 not found: ID does not exist" containerID="30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982" May 28 20:56:07.260699 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.260621 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982"} err="failed to get container status \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": rpc error: code = NotFound desc = could not find container \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": container with ID starting with 30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982 not found: ID does not exist" May 28 20:56:07.260699 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.260643 2563 scope.go:117] "RemoveContainer" containerID="4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488" May 28 20:56:07.260920 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.260901 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488"} err="failed to get container status \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": rpc error: code = NotFound desc = could not find container \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": container with ID starting with 4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488 not found: ID does not exist" May 28 20:56:07.260963 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.260922 2563 scope.go:117] "RemoveContainer" containerID="9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c" May 28 20:56:07.261546 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.261526 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c"} err="failed to get container status \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": rpc error: code = NotFound desc = could not find container \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": container with ID starting with 9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c not found: ID does not exist" May 28 20:56:07.261653 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.261566 2563 scope.go:117] "RemoveContainer" containerID="99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604" May 28 20:56:07.261790 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.261773 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" May 28 20:56:07.261890 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.261849 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604"} err="failed to get container status \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": rpc error: code = NotFound desc = could not find container \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": container with ID starting with 99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604 not found: ID does not exist" May 28 20:56:07.261890 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.261872 2563 scope.go:117] "RemoveContainer" containerID="2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508" May 28 20:56:07.262009 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.261894 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" May 28 20:56:07.262009 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.261976 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" May 28 20:56:07.262120 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262088 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" May 28 20:56:07.262120 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262100 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" May 28 20:56:07.262120 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262106 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" May 28 20:56:07.262303 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262194 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508"} err="failed to get container status \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": rpc error: code = NotFound desc = could not find container \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": container with ID starting with 2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508 not found: ID does not exist" May 28 20:56:07.262303 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262224 2563 scope.go:117] "RemoveContainer" containerID="c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc" May 28 20:56:07.262303 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262270 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6sdtw\"" May 28 20:56:07.262489 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262394 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" May 28 20:56:07.262489 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262467 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc"} err="failed to get container status \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": rpc error: code = NotFound desc = could not find container \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": container with ID starting with c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc not found: ID does not exist" May 28 20:56:07.262601 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262493 2563 scope.go:117] "RemoveContainer" containerID="f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af" May 28 20:56:07.262673 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262654 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" May 28 20:56:07.262738 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262685 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" May 28 20:56:07.262738 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262697 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" May 28 20:56:07.262858 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262781 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ab2jvkn6ueohg\"" May 28 20:56:07.262858 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262795 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af"} err="failed to get container status \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": rpc error: code = NotFound desc = could not find container \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": container with ID starting with f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af not found: ID does not exist" May 28 20:56:07.262858 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262820 2563 scope.go:117] "RemoveContainer" containerID="30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982" May 28 20:56:07.262972 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.262880 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" May 28 20:56:07.263145 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.263109 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982"} err="failed to get container status \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": rpc error: code = NotFound desc = could not find container \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": container with ID starting with 30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982 not found: ID does not exist" May 28 20:56:07.263417 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.263148 2563 scope.go:117] "RemoveContainer" containerID="4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488" May 28 20:56:07.263486 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.263464 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488"} err="failed to get container status \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": rpc error: code = NotFound desc = could not find container \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": container with ID starting with 4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488 not found: ID does not exist" May 28 20:56:07.263638 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.263491 2563 scope.go:117] "RemoveContainer" containerID="9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c" May 28 20:56:07.264289 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.264158 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c"} err="failed to get container status \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": rpc error: code = NotFound desc = could not find container \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": container with ID starting with 9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c not found: ID does not exist" May 28 20:56:07.264289 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.264209 2563 scope.go:117] "RemoveContainer" containerID="99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604" May 28 20:56:07.264874 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.264596 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604"} err="failed to get container status \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": rpc error: code = NotFound desc = could not find container \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": container with ID starting with 99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604 not found: ID does not exist" May 28 20:56:07.264874 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.264629 2563 scope.go:117] "RemoveContainer" containerID="2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508" May 28 20:56:07.265022 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.264889 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508"} err="failed to get container status \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": rpc error: code = NotFound desc = could not find container \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": container with ID starting with 2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508 not found: ID does not exist" May 28 20:56:07.265022 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.264921 2563 scope.go:117] "RemoveContainer" containerID="c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc" May 28 20:56:07.265356 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.265317 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc"} err="failed to get container status \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": rpc error: code = NotFound desc = could not find container \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": container with ID starting with c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc not found: ID does not exist" May 28 20:56:07.265449 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.265361 2563 scope.go:117] "RemoveContainer" containerID="f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af" May 28 20:56:07.265676 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.265655 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af"} err="failed to get container status \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": rpc error: code = NotFound desc = could not find container \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": container with ID starting with f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af not found: ID does not exist" May 28 20:56:07.265761 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.265678 2563 scope.go:117] "RemoveContainer" containerID="30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982" May 28 20:56:07.266015 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.265985 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982"} err="failed to get container status \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": rpc error: code = NotFound desc = could not find container \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": container with ID starting with 30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982 not found: ID does not exist" May 28 20:56:07.266104 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.266017 2563 scope.go:117] "RemoveContainer" containerID="4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488" May 28 20:56:07.266346 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.266323 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488"} err="failed to get container status \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": rpc error: code = NotFound desc = could not find container \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": container with ID starting with 4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488 not found: ID does not exist" May 28 20:56:07.266346 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.266342 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" May 28 20:56:07.266537 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.266345 2563 scope.go:117] "RemoveContainer" containerID="9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c" May 28 20:56:07.268133 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.268068 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c"} err="failed to get container status \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": rpc error: code = NotFound desc = could not find container \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": container with ID starting with 9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c not found: ID does not exist" May 28 20:56:07.268249 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.268135 2563 scope.go:117] "RemoveContainer" containerID="99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604" May 28 20:56:07.268596 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.268569 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604"} err="failed to get container status \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": rpc error: code = NotFound desc = could not find container \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": container with ID starting with 99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604 not found: ID does not exist" May 28 20:56:07.268682 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.268597 2563 scope.go:117] "RemoveContainer" containerID="2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508" May 28 20:56:07.273281 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.272751 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508"} err="failed to get container status \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": rpc error: code = NotFound desc = could not find container \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": container with ID starting with 2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508 not found: ID does not exist" May 28 20:56:07.273281 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.272777 2563 scope.go:117] "RemoveContainer" containerID="c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc" May 28 20:56:07.273587 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.273479 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc"} err="failed to get container status \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": rpc error: code = NotFound desc = could not find container \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": container with ID starting with c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc not found: ID does not exist" May 28 20:56:07.273587 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.273506 2563 scope.go:117] "RemoveContainer" containerID="f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af" May 28 20:56:07.273960 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.273818 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af"} err="failed to get container status \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": rpc error: code = NotFound desc = could not find container \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": container with ID starting with f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af not found: ID does not exist" May 28 20:56:07.273960 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.273843 2563 scope.go:117] "RemoveContainer" containerID="30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982" May 28 20:56:07.274253 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.274120 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" May 28 20:56:07.274319 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.274241 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982"} err="failed to get container status \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": rpc error: code = NotFound desc = could not find container \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": container with ID starting with 30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982 not found: ID does not exist" May 28 20:56:07.274429 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.274393 2563 scope.go:117] "RemoveContainer" containerID="4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488" May 28 20:56:07.274749 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.274725 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488"} err="failed to get container status \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": rpc error: code = NotFound desc = could not find container \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": container with ID starting with 4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488 not found: ID does not exist" May 28 20:56:07.274829 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.274761 2563 scope.go:117] "RemoveContainer" containerID="9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c" May 28 20:56:07.275034 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.275012 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c"} err="failed to get container status \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": rpc error: code = NotFound desc = could not find container \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": container with ID starting with 9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c not found: ID does not exist" May 28 20:56:07.275034 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.275034 2563 scope.go:117] "RemoveContainer" containerID="99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604" May 28 20:56:07.275325 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.275302 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604"} err="failed to get container status \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": rpc error: code = NotFound desc = could not find container \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": container with ID starting with 99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604 not found: ID does not exist" May 28 20:56:07.275429 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.275326 2563 scope.go:117] "RemoveContainer" containerID="2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508" May 28 20:56:07.275429 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.275339 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 20:56:07.276134 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.275963 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508"} err="failed to get container status \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": rpc error: code = NotFound desc = could not find container \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": container with ID starting with 2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508 not found: ID does not exist" May 28 20:56:07.276134 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.275993 2563 scope.go:117] "RemoveContainer" containerID="c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc" May 28 20:56:07.276333 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.276307 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc"} err="failed to get container status \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": rpc error: code = NotFound desc = could not find container \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": container with ID starting with c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc not found: ID does not exist" May 28 20:56:07.276396 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.276333 2563 scope.go:117] "RemoveContainer" containerID="f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af" May 28 20:56:07.276588 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.276568 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af"} err="failed to get container status \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": rpc error: code = NotFound desc = could not find container \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": container with ID starting with f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af not found: ID does not exist" May 28 20:56:07.276639 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.276589 2563 scope.go:117] "RemoveContainer" containerID="30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982" May 28 20:56:07.276806 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.276791 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982"} err="failed to get container status \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": rpc error: code = NotFound desc = could not find container \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": container with ID starting with 30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982 not found: ID does not exist" May 28 20:56:07.276852 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.276806 2563 scope.go:117] "RemoveContainer" containerID="4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488" May 28 20:56:07.276989 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.276974 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488"} err="failed to get container status \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": rpc error: code = NotFound desc = could not find container \"4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488\": container with ID starting with 4332362e6e5a4656cb370f24047e338d9de20483e9d36bf1db905603a141e488 not found: ID does not exist" May 28 20:56:07.276989 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.276988 2563 scope.go:117] "RemoveContainer" containerID="9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c" May 28 20:56:07.277240 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.277218 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c"} err="failed to get container status \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": rpc error: code = NotFound desc = could not find container \"9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c\": container with ID starting with 9d48dd10c8392b47b5d3e738d999c93e79069a6b8077bb34a2c6b24605907e6c not found: ID does not exist" May 28 20:56:07.277240 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.277240 2563 scope.go:117] "RemoveContainer" containerID="99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604" May 28 20:56:07.277515 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.277486 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604"} err="failed to get container status \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": rpc error: code = NotFound desc = could not find container \"99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604\": container with ID starting with 99450bbde8ec83822c80cae26da0bfca35c0279336d6efa0c587417b7e7c8604 not found: ID does not exist" May 28 20:56:07.277617 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.277516 2563 scope.go:117] "RemoveContainer" containerID="2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508" May 28 20:56:07.277788 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.277763 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508"} err="failed to get container status \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": rpc error: code = NotFound desc = could not find container \"2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508\": container with ID starting with 2ddd7bd55b234971e5ddb791249998ce0c48f176b2261cbd136d98532f811508 not found: ID does not exist" May 28 20:56:07.277861 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.277790 2563 scope.go:117] "RemoveContainer" containerID="c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc" May 28 20:56:07.278092 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.278073 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc"} err="failed to get container status \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": rpc error: code = NotFound desc = could not find container \"c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc\": container with ID starting with c2262814e73e62d9566fa603bea5602357ca519b42deef42710aee088379bccc not found: ID does not exist" May 28 20:56:07.278185 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.278093 2563 scope.go:117] "RemoveContainer" containerID="f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af" May 28 20:56:07.278415 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.278374 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af"} err="failed to get container status \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": rpc error: code = NotFound desc = could not find container \"f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af\": container with ID starting with f47744d2d29a581c97b84c9be690fab4629d3ba08cc19b01755415d93d7768af not found: ID does not exist" May 28 20:56:07.278495 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.278417 2563 scope.go:117] "RemoveContainer" containerID="30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982" May 28 20:56:07.278709 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.278690 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982"} err="failed to get container status \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": rpc error: code = NotFound desc = could not find container \"30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982\": container with ID starting with 30951a715c0d3979e619943660754f7cad46b95614af22c539cbf7226ca90982 not found: ID does not exist" May 28 20:56:07.356448 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356418 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356597 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356453 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356597 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356472 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356597 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356489 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356597 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356551 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356765 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356612 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356765 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356635 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79dzv\" (UniqueName: \"kubernetes.io/projected/6a2ec453-3aea-4cb2-971f-b962455fe573-kube-api-access-79dzv\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356765 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356709 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356765 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356750 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-config\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356765 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356766 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-web-config\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356941 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356782 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6a2ec453-3aea-4cb2-971f-b962455fe573-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356941 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356797 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356941 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356816 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a2ec453-3aea-4cb2-971f-b962455fe573-config-out\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356941 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356941 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356856 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356941 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356874 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a2ec453-3aea-4cb2-971f-b962455fe573-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356941 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356897 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.356941 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.356911 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458253 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458181 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458253 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458220 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458253 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458242 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458262 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458280 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458294 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-79dzv\" (UniqueName: \"kubernetes.io/projected/6a2ec453-3aea-4cb2-971f-b962455fe573-kube-api-access-79dzv\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458325 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458356 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-config\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458381 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-web-config\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458404 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6a2ec453-3aea-4cb2-971f-b962455fe573-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458433 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458467 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a2ec453-3aea-4cb2-971f-b962455fe573-config-out\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.458504 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458493 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.459032 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458531 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.459032 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458559 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a2ec453-3aea-4cb2-971f-b962455fe573-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.459032 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458600 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.459032 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458626 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.459032 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.458684 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.459511 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.459487 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.460606 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.460265 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.461409 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.461291 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.461409 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.461309 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.461409 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.461370 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.461409 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.461389 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.461695 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.461454 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.461695 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.461576 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6a2ec453-3aea-4cb2-971f-b962455fe573-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.462638 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.462028 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.462638 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.462545 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-web-config\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.462893 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.462870 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.463232 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.463212 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.463710 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.463686 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a2ec453-3aea-4cb2-971f-b962455fe573-config-out\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.463913 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.463882 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-config\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.464088 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.464071 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6a2ec453-3aea-4cb2-971f-b962455fe573-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.464158 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.464120 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a2ec453-3aea-4cb2-971f-b962455fe573-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.464836 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.464817 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6a2ec453-3aea-4cb2-971f-b962455fe573-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.466488 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.466472 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79dzv\" (UniqueName: \"kubernetes.io/projected/6a2ec453-3aea-4cb2-971f-b962455fe573-kube-api-access-79dzv\") pod \"prometheus-k8s-0\" (UID: \"6a2ec453-3aea-4cb2-971f-b962455fe573\") " pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.577351 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.577315 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:56:07.719497 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:07.719475 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] May 28 20:56:07.721403 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:56:07.721375 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a2ec453_3aea_4cb2_971f_b962455fe573.slice/crio-5399702789de3e286072d31d09abba15b719a81b0bdfe74513cbe987e1cb2925 WatchSource:0}: Error finding container 5399702789de3e286072d31d09abba15b719a81b0bdfe74513cbe987e1cb2925: Status 404 returned error can't find the container with id 5399702789de3e286072d31d09abba15b719a81b0bdfe74513cbe987e1cb2925 May 28 20:56:08.199037 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:08.199003 2563 generic.go:358] "Generic (PLEG): container finished" podID="6a2ec453-3aea-4cb2-971f-b962455fe573" containerID="7667d4791dd72b20d9099ff94dacbab84b10b09bfeaf214e01be57d024462601" exitCode=0 May 28 20:56:08.199255 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:08.199071 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a2ec453-3aea-4cb2-971f-b962455fe573","Type":"ContainerDied","Data":"7667d4791dd72b20d9099ff94dacbab84b10b09bfeaf214e01be57d024462601"} May 28 20:56:08.199255 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:08.199090 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a2ec453-3aea-4cb2-971f-b962455fe573","Type":"ContainerStarted","Data":"5399702789de3e286072d31d09abba15b719a81b0bdfe74513cbe987e1cb2925"} May 28 20:56:08.468721 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:08.468228 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb5f8c5e-e737-4e39-baf5-b0bc38698331" path="/var/lib/kubelet/pods/bb5f8c5e-e737-4e39-baf5-b0bc38698331/volumes" May 28 20:56:09.205180 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:09.205147 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a2ec453-3aea-4cb2-971f-b962455fe573","Type":"ContainerStarted","Data":"eadbabb46b4fe331623f07ade340d2f9713cfa2fe2835398add894061af1df80"} May 28 20:56:09.205326 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:09.205194 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a2ec453-3aea-4cb2-971f-b962455fe573","Type":"ContainerStarted","Data":"90df1b7b2e9631e8e058442b0756fb7df1280467f47b41ea6bc7854ad0dbc0c4"} May 28 20:56:09.205326 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:09.205207 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a2ec453-3aea-4cb2-971f-b962455fe573","Type":"ContainerStarted","Data":"40adbb1fff46784ae25d42ae4a40ecaeac28cbad1df401f254990a0cb431c54e"} May 28 20:56:09.205326 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:09.205216 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a2ec453-3aea-4cb2-971f-b962455fe573","Type":"ContainerStarted","Data":"5e476873aa4044c4f8bad9dc52857e4ba17f4991a466cf6b2066517528d1c29d"} May 28 20:56:09.205326 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:09.205225 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a2ec453-3aea-4cb2-971f-b962455fe573","Type":"ContainerStarted","Data":"e6380bf922cc79e6747835c0e944c4229f4986202eeae45281c4cac0082c2980"} May 28 20:56:09.205326 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:09.205233 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a2ec453-3aea-4cb2-971f-b962455fe573","Type":"ContainerStarted","Data":"c1abf80205589e7e6d474ed4fd28b5e5de07213066c3cf75f90a1dd58b0db442"} May 28 20:56:09.231647 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:09.231595 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.231577403 podStartE2EDuration="2.231577403s" podCreationTimestamp="2026-05-28 20:56:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 20:56:09.229779196 +0000 UTC m=+219.347172781" watchObservedRunningTime="2026-05-28 20:56:09.231577403 +0000 UTC m=+219.348970989" May 28 20:56:12.578301 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:56:12.578270 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:57:07.578372 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:57:07.578331 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:57:07.593933 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:57:07.593901 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:57:08.397324 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:57:08.397297 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" May 28 20:57:30.392705 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:57:30.392675 2563 kubelet.go:1628] "Image garbage collection succeeded" May 28 20:59:44.175069 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.175029 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq"] May 28 20:59:44.177368 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.177346 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:44.179389 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.179368 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 28 20:59:44.179970 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.179948 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 28 20:59:44.180105 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.180081 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s2tr4\"" May 28 20:59:44.180230 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.180114 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 28 20:59:44.180281 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.180234 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 28 20:59:44.188781 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.188760 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq"] May 28 20:59:44.276356 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.276325 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9792bf4-afb0-42af-b33d-4cb081565ea1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-gpspq\" (UID: \"d9792bf4-afb0-42af-b33d-4cb081565ea1\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:44.276529 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.276386 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smbd9\" (UniqueName: \"kubernetes.io/projected/d9792bf4-afb0-42af-b33d-4cb081565ea1-kube-api-access-smbd9\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-gpspq\" (UID: \"d9792bf4-afb0-42af-b33d-4cb081565ea1\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:44.276529 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.276413 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9792bf4-afb0-42af-b33d-4cb081565ea1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-gpspq\" (UID: \"d9792bf4-afb0-42af-b33d-4cb081565ea1\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:44.377584 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.377551 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9792bf4-afb0-42af-b33d-4cb081565ea1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-gpspq\" (UID: \"d9792bf4-afb0-42af-b33d-4cb081565ea1\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:44.377757 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.377694 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-smbd9\" (UniqueName: \"kubernetes.io/projected/d9792bf4-afb0-42af-b33d-4cb081565ea1-kube-api-access-smbd9\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-gpspq\" (UID: \"d9792bf4-afb0-42af-b33d-4cb081565ea1\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:44.377757 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.377735 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9792bf4-afb0-42af-b33d-4cb081565ea1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-gpspq\" (UID: \"d9792bf4-afb0-42af-b33d-4cb081565ea1\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:44.379886 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.379865 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d9792bf4-afb0-42af-b33d-4cb081565ea1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-gpspq\" (UID: \"d9792bf4-afb0-42af-b33d-4cb081565ea1\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:44.379999 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.379982 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9792bf4-afb0-42af-b33d-4cb081565ea1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-gpspq\" (UID: \"d9792bf4-afb0-42af-b33d-4cb081565ea1\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:44.385638 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.385615 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smbd9\" (UniqueName: \"kubernetes.io/projected/d9792bf4-afb0-42af-b33d-4cb081565ea1-kube-api-access-smbd9\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-gpspq\" (UID: \"d9792bf4-afb0-42af-b33d-4cb081565ea1\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:44.488395 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.488373 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:44.620020 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.619994 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq"] May 28 20:59:44.622843 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:59:44.622817 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9792bf4_afb0_42af_b33d_4cb081565ea1.slice/crio-ffdecfee0bd601a2078dd4f6681b27260c0fe950531baccb2ad1abd2ba7c607c WatchSource:0}: Error finding container ffdecfee0bd601a2078dd4f6681b27260c0fe950531baccb2ad1abd2ba7c607c: Status 404 returned error can't find the container with id ffdecfee0bd601a2078dd4f6681b27260c0fe950531baccb2ad1abd2ba7c607c May 28 20:59:44.624493 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.624465 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 20:59:44.814837 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:44.814761 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" event={"ID":"d9792bf4-afb0-42af-b33d-4cb081565ea1","Type":"ContainerStarted","Data":"ffdecfee0bd601a2078dd4f6681b27260c0fe950531baccb2ad1abd2ba7c607c"} May 28 20:59:47.828179 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:47.828138 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" event={"ID":"d9792bf4-afb0-42af-b33d-4cb081565ea1","Type":"ContainerStarted","Data":"739c6399473de05ef6d887fc74d52a770286ef766d3d5810a35b924f72ccc5b5"} May 28 20:59:47.828566 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:47.828220 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:47.848157 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:47.848106 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" podStartSLOduration=1.150256699 podStartE2EDuration="3.848092012s" podCreationTimestamp="2026-05-28 20:59:44 +0000 UTC" firstStartedPulling="2026-05-28 20:59:44.624584678 +0000 UTC m=+434.741978242" lastFinishedPulling="2026-05-28 20:59:47.322419981 +0000 UTC m=+437.439813555" observedRunningTime="2026-05-28 20:59:47.847281588 +0000 UTC m=+437.964675182" watchObservedRunningTime="2026-05-28 20:59:47.848092012 +0000 UTC m=+437.965485598" May 28 20:59:54.907340 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:54.907306 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q"] May 28 20:59:54.910421 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:54.910406 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:54.917972 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:54.917954 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jrrkq\"" May 28 20:59:54.924275 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:54.924249 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" May 28 20:59:54.924420 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:54.924296 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" May 28 20:59:54.924420 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:54.924329 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" May 28 20:59:54.924420 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:54.924357 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" May 28 20:59:54.924565 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:54.924420 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" May 28 20:59:54.931884 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:54.931864 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q"] May 28 20:59:55.072994 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.068588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-metrics-cert\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.072994 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.068668 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-manager-config\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.072994 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.068737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-cert\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.072994 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.068809 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdjh5\" (UniqueName: \"kubernetes.io/projected/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-kube-api-access-sdjh5\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.169693 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.169606 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-metrics-cert\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.169693 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.169649 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-manager-config\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.169693 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.169682 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-cert\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.169992 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.169723 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sdjh5\" (UniqueName: \"kubernetes.io/projected/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-kube-api-access-sdjh5\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.170412 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.170389 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-manager-config\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.172116 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.172089 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-metrics-cert\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.172228 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.172191 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-cert\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.177785 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.177760 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdjh5\" (UniqueName: \"kubernetes.io/projected/ab3caa90-b7f3-4037-9065-cebaf9b6f1c4-kube-api-access-sdjh5\") pod \"lws-controller-manager-8459dbdd59-lxd2q\" (UID: \"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4\") " pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.219630 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.219608 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:55.348993 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.348874 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q"] May 28 20:59:55.351725 ip-10-0-140-94 kubenswrapper[2563]: W0528 20:59:55.351686 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab3caa90_b7f3_4037_9065_cebaf9b6f1c4.slice/crio-2890e42b82bce677935bfc62a2fc2469522719ec4723e4dd227945fca2669eb2 WatchSource:0}: Error finding container 2890e42b82bce677935bfc62a2fc2469522719ec4723e4dd227945fca2669eb2: Status 404 returned error can't find the container with id 2890e42b82bce677935bfc62a2fc2469522719ec4723e4dd227945fca2669eb2 May 28 20:59:55.853958 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:55.853920 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" event={"ID":"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4","Type":"ContainerStarted","Data":"2890e42b82bce677935bfc62a2fc2469522719ec4723e4dd227945fca2669eb2"} May 28 20:59:58.833429 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:58.833399 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-gpspq" May 28 20:59:59.868745 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:59.868707 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" event={"ID":"ab3caa90-b7f3-4037-9065-cebaf9b6f1c4","Type":"ContainerStarted","Data":"8675ee9319bb2517b41afb54fdf2689f34f77c85fac5f02b3d7ea0dd4d2c6a3b"} May 28 20:59:59.869193 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:59.868818 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 20:59:59.885816 ip-10-0-140-94 kubenswrapper[2563]: I0528 20:59:59.885760 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" podStartSLOduration=2.007232411 podStartE2EDuration="5.885743315s" podCreationTimestamp="2026-05-28 20:59:54 +0000 UTC" firstStartedPulling="2026-05-28 20:59:55.353356304 +0000 UTC m=+445.470749867" lastFinishedPulling="2026-05-28 20:59:59.231867204 +0000 UTC m=+449.349260771" observedRunningTime="2026-05-28 20:59:59.884636085 +0000 UTC m=+450.002029670" watchObservedRunningTime="2026-05-28 20:59:59.885743315 +0000 UTC m=+450.003136901" May 28 21:00:02.950289 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:02.950253 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs"] May 28 21:00:02.953662 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:02.953641 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" May 28 21:00:02.955588 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:02.955567 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 28 21:00:02.956292 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:02.956274 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dtjbd\"" May 28 21:00:02.956393 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:02.956329 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" May 28 21:00:02.962126 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:02.962101 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs"] May 28 21:00:03.138008 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.137975 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af2b87fd-53fb-4a13-8667-c2e686fd56f6-tls-certs\") pod \"kube-auth-proxy-774cfddf4d-2hcfs\" (UID: \"af2b87fd-53fb-4a13-8667-c2e686fd56f6\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" May 28 21:00:03.138194 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.138037 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vthj\" (UniqueName: \"kubernetes.io/projected/af2b87fd-53fb-4a13-8667-c2e686fd56f6-kube-api-access-7vthj\") pod \"kube-auth-proxy-774cfddf4d-2hcfs\" (UID: \"af2b87fd-53fb-4a13-8667-c2e686fd56f6\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" May 28 21:00:03.138194 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.138130 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af2b87fd-53fb-4a13-8667-c2e686fd56f6-tmp\") pod \"kube-auth-proxy-774cfddf4d-2hcfs\" (UID: \"af2b87fd-53fb-4a13-8667-c2e686fd56f6\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" May 28 21:00:03.238974 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.238935 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7vthj\" (UniqueName: \"kubernetes.io/projected/af2b87fd-53fb-4a13-8667-c2e686fd56f6-kube-api-access-7vthj\") pod \"kube-auth-proxy-774cfddf4d-2hcfs\" (UID: \"af2b87fd-53fb-4a13-8667-c2e686fd56f6\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" May 28 21:00:03.239149 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.238988 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af2b87fd-53fb-4a13-8667-c2e686fd56f6-tmp\") pod \"kube-auth-proxy-774cfddf4d-2hcfs\" (UID: \"af2b87fd-53fb-4a13-8667-c2e686fd56f6\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" May 28 21:00:03.239149 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.239030 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af2b87fd-53fb-4a13-8667-c2e686fd56f6-tls-certs\") pod \"kube-auth-proxy-774cfddf4d-2hcfs\" (UID: \"af2b87fd-53fb-4a13-8667-c2e686fd56f6\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" May 28 21:00:03.241410 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.241377 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af2b87fd-53fb-4a13-8667-c2e686fd56f6-tmp\") pod \"kube-auth-proxy-774cfddf4d-2hcfs\" (UID: \"af2b87fd-53fb-4a13-8667-c2e686fd56f6\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" May 28 21:00:03.241512 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.241494 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af2b87fd-53fb-4a13-8667-c2e686fd56f6-tls-certs\") pod \"kube-auth-proxy-774cfddf4d-2hcfs\" (UID: \"af2b87fd-53fb-4a13-8667-c2e686fd56f6\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" May 28 21:00:03.245990 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.245965 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vthj\" (UniqueName: \"kubernetes.io/projected/af2b87fd-53fb-4a13-8667-c2e686fd56f6-kube-api-access-7vthj\") pod \"kube-auth-proxy-774cfddf4d-2hcfs\" (UID: \"af2b87fd-53fb-4a13-8667-c2e686fd56f6\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" May 28 21:00:03.263858 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.263835 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" May 28 21:00:03.393371 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.393337 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs"] May 28 21:00:03.397764 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:00:03.397738 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf2b87fd_53fb_4a13_8667_c2e686fd56f6.slice/crio-27c929e87fc4750dc75d1f8846200ce8e1cc70c7eb85839b5f76a13fc3d091e0 WatchSource:0}: Error finding container 27c929e87fc4750dc75d1f8846200ce8e1cc70c7eb85839b5f76a13fc3d091e0: Status 404 returned error can't find the container with id 27c929e87fc4750dc75d1f8846200ce8e1cc70c7eb85839b5f76a13fc3d091e0 May 28 21:00:03.882680 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:03.882646 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" event={"ID":"af2b87fd-53fb-4a13-8667-c2e686fd56f6","Type":"ContainerStarted","Data":"27c929e87fc4750dc75d1f8846200ce8e1cc70c7eb85839b5f76a13fc3d091e0"} May 28 21:00:07.897804 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:07.897762 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" event={"ID":"af2b87fd-53fb-4a13-8667-c2e686fd56f6","Type":"ContainerStarted","Data":"407d3bf29733ee2f21fb0c9bc193f9e4480b36ad5297ee9474fca1189867dacf"} May 28 21:00:07.912887 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:07.912837 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-774cfddf4d-2hcfs" podStartSLOduration=2.4585629239999998 podStartE2EDuration="5.912821091s" podCreationTimestamp="2026-05-28 21:00:02 +0000 UTC" firstStartedPulling="2026-05-28 21:00:03.399636312 +0000 UTC m=+453.517029875" lastFinishedPulling="2026-05-28 21:00:06.853894465 +0000 UTC m=+456.971288042" observedRunningTime="2026-05-28 21:00:07.91154304 +0000 UTC m=+458.028936634" watchObservedRunningTime="2026-05-28 21:00:07.912821091 +0000 UTC m=+458.030214677" May 28 21:00:10.874311 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:00:10.874280 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8459dbdd59-lxd2q" May 28 21:01:44.774470 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:44.774391 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86"] May 28 21:01:44.776676 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:44.776659 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" May 28 21:01:44.778949 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:44.778928 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 28 21:01:44.779033 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:44.778990 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 28 21:01:44.779449 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:44.779434 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rw484\"" May 28 21:01:44.786814 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:44.786792 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86"] May 28 21:01:44.845628 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:44.845600 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfgtb\" (UniqueName: \"kubernetes.io/projected/ce99d7ba-0774-488a-9ecb-193452e09797-kube-api-access-lfgtb\") pod \"limitador-operator-controller-manager-85c4996f8c-65n86\" (UID: \"ce99d7ba-0774-488a-9ecb-193452e09797\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" May 28 21:01:44.946281 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:44.946237 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lfgtb\" (UniqueName: \"kubernetes.io/projected/ce99d7ba-0774-488a-9ecb-193452e09797-kube-api-access-lfgtb\") pod \"limitador-operator-controller-manager-85c4996f8c-65n86\" (UID: \"ce99d7ba-0774-488a-9ecb-193452e09797\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" May 28 21:01:44.954181 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:44.954143 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfgtb\" (UniqueName: \"kubernetes.io/projected/ce99d7ba-0774-488a-9ecb-193452e09797-kube-api-access-lfgtb\") pod \"limitador-operator-controller-manager-85c4996f8c-65n86\" (UID: \"ce99d7ba-0774-488a-9ecb-193452e09797\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" May 28 21:01:45.087287 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:45.087223 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" May 28 21:01:45.224553 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:45.224533 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86"] May 28 21:01:45.226841 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:01:45.226814 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce99d7ba_0774_488a_9ecb_193452e09797.slice/crio-30e556952e0579213b00b3036031042ce5da553252fc533b7c7612407b3db521 WatchSource:0}: Error finding container 30e556952e0579213b00b3036031042ce5da553252fc533b7c7612407b3db521: Status 404 returned error can't find the container with id 30e556952e0579213b00b3036031042ce5da553252fc533b7c7612407b3db521 May 28 21:01:46.215708 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:46.215677 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" event={"ID":"ce99d7ba-0774-488a-9ecb-193452e09797","Type":"ContainerStarted","Data":"30e556952e0579213b00b3036031042ce5da553252fc533b7c7612407b3db521"} May 28 21:01:48.223865 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:48.223830 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" event={"ID":"ce99d7ba-0774-488a-9ecb-193452e09797","Type":"ContainerStarted","Data":"6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d"} May 28 21:01:48.224338 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:48.223953 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" May 28 21:01:48.252744 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:48.252694 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" podStartSLOduration=2.08203774 podStartE2EDuration="4.252679336s" podCreationTimestamp="2026-05-28 21:01:44 +0000 UTC" firstStartedPulling="2026-05-28 21:01:45.228639459 +0000 UTC m=+555.346033025" lastFinishedPulling="2026-05-28 21:01:47.399281055 +0000 UTC m=+557.516674621" observedRunningTime="2026-05-28 21:01:48.251310877 +0000 UTC m=+558.368704454" watchObservedRunningTime="2026-05-28 21:01:48.252679336 +0000 UTC m=+558.370072922" May 28 21:01:57.613726 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.613694 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86"] May 28 21:01:57.614130 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.613923 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" podUID="ce99d7ba-0774-488a-9ecb-193452e09797" containerName="manager" containerID="cri-o://6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d" gracePeriod=2 May 28 21:01:57.615841 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.615814 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" May 28 21:01:57.625964 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.625929 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86"] May 28 21:01:57.627879 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.627848 2563 status_manager.go:895] "Failed to get status for pod" podUID="ce99d7ba-0774-488a-9ecb-193452e09797" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" err="pods \"limitador-operator-controller-manager-85c4996f8c-65n86\" is forbidden: User \"system:node:ip-10-0-140-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-94.ec2.internal' and this object" May 28 21:01:57.642430 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.642405 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz"] May 28 21:01:57.642760 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.642746 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce99d7ba-0774-488a-9ecb-193452e09797" containerName="manager" May 28 21:01:57.642760 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.642761 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce99d7ba-0774-488a-9ecb-193452e09797" containerName="manager" May 28 21:01:57.642875 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.642833 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="ce99d7ba-0774-488a-9ecb-193452e09797" containerName="manager" May 28 21:01:57.644620 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.644604 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz" May 28 21:01:57.659257 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.659227 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz"] May 28 21:01:57.672506 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.672472 2563 status_manager.go:895] "Failed to get status for pod" podUID="ce99d7ba-0774-488a-9ecb-193452e09797" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" err="pods \"limitador-operator-controller-manager-85c4996f8c-65n86\" is forbidden: User \"system:node:ip-10-0-140-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-94.ec2.internal' and this object" May 28 21:01:57.760039 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.760003 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf6hk\" (UniqueName: \"kubernetes.io/projected/c8941719-cbf5-42d7-a7c1-0f2dc49e456a-kube-api-access-gf6hk\") pod \"limitador-operator-controller-manager-85c4996f8c-g6mjz\" (UID: \"c8941719-cbf5-42d7-a7c1-0f2dc49e456a\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz" May 28 21:01:57.841529 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.841508 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" May 28 21:01:57.843742 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.843718 2563 status_manager.go:895] "Failed to get status for pod" podUID="ce99d7ba-0774-488a-9ecb-193452e09797" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" err="pods \"limitador-operator-controller-manager-85c4996f8c-65n86\" is forbidden: User \"system:node:ip-10-0-140-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-94.ec2.internal' and this object" May 28 21:01:57.861268 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.861247 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gf6hk\" (UniqueName: \"kubernetes.io/projected/c8941719-cbf5-42d7-a7c1-0f2dc49e456a-kube-api-access-gf6hk\") pod \"limitador-operator-controller-manager-85c4996f8c-g6mjz\" (UID: \"c8941719-cbf5-42d7-a7c1-0f2dc49e456a\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz" May 28 21:01:57.870350 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.870301 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf6hk\" (UniqueName: \"kubernetes.io/projected/c8941719-cbf5-42d7-a7c1-0f2dc49e456a-kube-api-access-gf6hk\") pod \"limitador-operator-controller-manager-85c4996f8c-g6mjz\" (UID: \"c8941719-cbf5-42d7-a7c1-0f2dc49e456a\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz" May 28 21:01:57.961738 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.961716 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfgtb\" (UniqueName: \"kubernetes.io/projected/ce99d7ba-0774-488a-9ecb-193452e09797-kube-api-access-lfgtb\") pod \"ce99d7ba-0774-488a-9ecb-193452e09797\" (UID: \"ce99d7ba-0774-488a-9ecb-193452e09797\") " May 28 21:01:57.963697 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.963675 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce99d7ba-0774-488a-9ecb-193452e09797-kube-api-access-lfgtb" (OuterVolumeSpecName: "kube-api-access-lfgtb") pod "ce99d7ba-0774-488a-9ecb-193452e09797" (UID: "ce99d7ba-0774-488a-9ecb-193452e09797"). InnerVolumeSpecName "kube-api-access-lfgtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 21:01:57.986871 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:57.986851 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz" May 28 21:01:58.062819 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.062789 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfgtb\" (UniqueName: \"kubernetes.io/projected/ce99d7ba-0774-488a-9ecb-193452e09797-kube-api-access-lfgtb\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 21:01:58.108537 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.108508 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz"] May 28 21:01:58.111112 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:01:58.111084 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8941719_cbf5_42d7_a7c1_0f2dc49e456a.slice/crio-0c2024e8044496e3fb88ff9e8966b90402f364aea8a653f92f3b9fb1477cf58a WatchSource:0}: Error finding container 0c2024e8044496e3fb88ff9e8966b90402f364aea8a653f92f3b9fb1477cf58a: Status 404 returned error can't find the container with id 0c2024e8044496e3fb88ff9e8966b90402f364aea8a653f92f3b9fb1477cf58a May 28 21:01:58.259370 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.259335 2563 generic.go:358] "Generic (PLEG): container finished" podID="ce99d7ba-0774-488a-9ecb-193452e09797" containerID="6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d" exitCode=0 May 28 21:01:58.259525 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.259388 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" May 28 21:01:58.259525 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.259429 2563 scope.go:117] "RemoveContainer" containerID="6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d" May 28 21:01:58.260845 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.260813 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz" event={"ID":"c8941719-cbf5-42d7-a7c1-0f2dc49e456a","Type":"ContainerStarted","Data":"1ec238eaa160c9e1dd917913b23db319b838376567aa7f08fc9ccc7d3fe4de29"} May 28 21:01:58.260957 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.260852 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz" event={"ID":"c8941719-cbf5-42d7-a7c1-0f2dc49e456a","Type":"ContainerStarted","Data":"0c2024e8044496e3fb88ff9e8966b90402f364aea8a653f92f3b9fb1477cf58a"} May 28 21:01:58.260957 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.260940 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz" May 28 21:01:58.263560 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.261741 2563 status_manager.go:895] "Failed to get status for pod" podUID="ce99d7ba-0774-488a-9ecb-193452e09797" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" err="pods \"limitador-operator-controller-manager-85c4996f8c-65n86\" is forbidden: User \"system:node:ip-10-0-140-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-94.ec2.internal' and this object" May 28 21:01:58.272691 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.272673 2563 scope.go:117] "RemoveContainer" containerID="6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d" May 28 21:01:58.272957 ip-10-0-140-94 kubenswrapper[2563]: E0528 21:01:58.272938 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d\": container with ID starting with 6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d not found: ID does not exist" containerID="6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d" May 28 21:01:58.273003 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.272965 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d"} err="failed to get container status \"6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d\": rpc error: code = NotFound desc = could not find container \"6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d\": container with ID starting with 6e327688737a1925d8d754431a62cad95d4800f2b52c9d78a9ed9a5944487d2d not found: ID does not exist" May 28 21:01:58.286629 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.286593 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz" podStartSLOduration=1.2865793810000001 podStartE2EDuration="1.286579381s" podCreationTimestamp="2026-05-28 21:01:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 21:01:58.285768764 +0000 UTC m=+568.403162361" watchObservedRunningTime="2026-05-28 21:01:58.286579381 +0000 UTC m=+568.403972968" May 28 21:01:58.288136 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.288114 2563 status_manager.go:895] "Failed to get status for pod" podUID="ce99d7ba-0774-488a-9ecb-193452e09797" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" err="pods \"limitador-operator-controller-manager-85c4996f8c-65n86\" is forbidden: User \"system:node:ip-10-0-140-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-94.ec2.internal' and this object" May 28 21:01:58.289900 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.289880 2563 status_manager.go:895] "Failed to get status for pod" podUID="ce99d7ba-0774-488a-9ecb-193452e09797" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-65n86" err="pods \"limitador-operator-controller-manager-85c4996f8c-65n86\" is forbidden: User \"system:node:ip-10-0-140-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-94.ec2.internal' and this object" May 28 21:01:58.467397 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:01:58.467320 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce99d7ba-0774-488a-9ecb-193452e09797" path="/var/lib/kubelet/pods/ce99d7ba-0774-488a-9ecb-193452e09797/volumes" May 28 21:02:09.267277 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:09.267249 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-g6mjz" May 28 21:02:35.941469 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:35.941436 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2lt72"] May 28 21:02:35.946825 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:35.946802 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-2lt72" May 28 21:02:35.948892 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:35.948872 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pkcfl\"" May 28 21:02:35.955552 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:35.955529 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2lt72"] May 28 21:02:36.073742 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.073707 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8jqh\" (UniqueName: \"kubernetes.io/projected/99414f6e-cc6a-4745-a6e4-eeb3c6995d16-kube-api-access-l8jqh\") pod \"authorino-f99f4b5cd-2lt72\" (UID: \"99414f6e-cc6a-4745-a6e4-eeb3c6995d16\") " pod="kuadrant-system/authorino-f99f4b5cd-2lt72" May 28 21:02:36.161403 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.161372 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-nd5cj"] May 28 21:02:36.164976 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.164960 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-nd5cj" May 28 21:02:36.172097 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.172075 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-nd5cj"] May 28 21:02:36.174053 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.174035 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l8jqh\" (UniqueName: \"kubernetes.io/projected/99414f6e-cc6a-4745-a6e4-eeb3c6995d16-kube-api-access-l8jqh\") pod \"authorino-f99f4b5cd-2lt72\" (UID: \"99414f6e-cc6a-4745-a6e4-eeb3c6995d16\") " pod="kuadrant-system/authorino-f99f4b5cd-2lt72" May 28 21:02:36.184814 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.184791 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8jqh\" (UniqueName: \"kubernetes.io/projected/99414f6e-cc6a-4745-a6e4-eeb3c6995d16-kube-api-access-l8jqh\") pod \"authorino-f99f4b5cd-2lt72\" (UID: \"99414f6e-cc6a-4745-a6e4-eeb3c6995d16\") " pod="kuadrant-system/authorino-f99f4b5cd-2lt72" May 28 21:02:36.266246 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.266220 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-2lt72" May 28 21:02:36.275156 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.275138 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrdp7\" (UniqueName: \"kubernetes.io/projected/3492266a-82d8-4c8b-9b49-da9a98735f6e-kube-api-access-lrdp7\") pod \"authorino-7498df8756-nd5cj\" (UID: \"3492266a-82d8-4c8b-9b49-da9a98735f6e\") " pod="kuadrant-system/authorino-7498df8756-nd5cj" May 28 21:02:36.376022 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.375990 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lrdp7\" (UniqueName: \"kubernetes.io/projected/3492266a-82d8-4c8b-9b49-da9a98735f6e-kube-api-access-lrdp7\") pod \"authorino-7498df8756-nd5cj\" (UID: \"3492266a-82d8-4c8b-9b49-da9a98735f6e\") " pod="kuadrant-system/authorino-7498df8756-nd5cj" May 28 21:02:36.383538 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.383509 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrdp7\" (UniqueName: \"kubernetes.io/projected/3492266a-82d8-4c8b-9b49-da9a98735f6e-kube-api-access-lrdp7\") pod \"authorino-7498df8756-nd5cj\" (UID: \"3492266a-82d8-4c8b-9b49-da9a98735f6e\") " pod="kuadrant-system/authorino-7498df8756-nd5cj" May 28 21:02:36.388915 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.388890 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2lt72"] May 28 21:02:36.392144 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:02:36.392118 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99414f6e_cc6a_4745_a6e4_eeb3c6995d16.slice/crio-f2b067fee061098a8abba079a8871151935d121c9f0826aeb98bc41cae4e2eb0 WatchSource:0}: Error finding container f2b067fee061098a8abba079a8871151935d121c9f0826aeb98bc41cae4e2eb0: Status 404 returned error can't find the container with id f2b067fee061098a8abba079a8871151935d121c9f0826aeb98bc41cae4e2eb0 May 28 21:02:36.473933 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.473908 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-nd5cj" May 28 21:02:36.594977 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:36.594937 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-nd5cj"] May 28 21:02:36.598731 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:02:36.598704 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3492266a_82d8_4c8b_9b49_da9a98735f6e.slice/crio-68ae751f1060723eece96c93fdea8d9b0542238426ac4c68c3b0f58006e43f17 WatchSource:0}: Error finding container 68ae751f1060723eece96c93fdea8d9b0542238426ac4c68c3b0f58006e43f17: Status 404 returned error can't find the container with id 68ae751f1060723eece96c93fdea8d9b0542238426ac4c68c3b0f58006e43f17 May 28 21:02:37.390674 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:37.390593 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-2lt72" event={"ID":"99414f6e-cc6a-4745-a6e4-eeb3c6995d16","Type":"ContainerStarted","Data":"f2b067fee061098a8abba079a8871151935d121c9f0826aeb98bc41cae4e2eb0"} May 28 21:02:37.396156 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:37.396098 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-nd5cj" event={"ID":"3492266a-82d8-4c8b-9b49-da9a98735f6e","Type":"ContainerStarted","Data":"68ae751f1060723eece96c93fdea8d9b0542238426ac4c68c3b0f58006e43f17"} May 28 21:02:40.409131 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:40.409090 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-2lt72" event={"ID":"99414f6e-cc6a-4745-a6e4-eeb3c6995d16","Type":"ContainerStarted","Data":"938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd"} May 28 21:02:40.410362 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:40.410326 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-nd5cj" event={"ID":"3492266a-82d8-4c8b-9b49-da9a98735f6e","Type":"ContainerStarted","Data":"313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8"} May 28 21:02:40.423534 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:40.423483 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-2lt72" podStartSLOduration=1.878284786 podStartE2EDuration="5.423473182s" podCreationTimestamp="2026-05-28 21:02:35 +0000 UTC" firstStartedPulling="2026-05-28 21:02:36.393270903 +0000 UTC m=+606.510664467" lastFinishedPulling="2026-05-28 21:02:39.93845929 +0000 UTC m=+610.055852863" observedRunningTime="2026-05-28 21:02:40.423127362 +0000 UTC m=+610.540520948" watchObservedRunningTime="2026-05-28 21:02:40.423473182 +0000 UTC m=+610.540866767" May 28 21:02:40.436463 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:40.436414 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-nd5cj" podStartSLOduration=1.086979805 podStartE2EDuration="4.436399072s" podCreationTimestamp="2026-05-28 21:02:36 +0000 UTC" firstStartedPulling="2026-05-28 21:02:36.599966908 +0000 UTC m=+606.717360472" lastFinishedPulling="2026-05-28 21:02:39.949386173 +0000 UTC m=+610.066779739" observedRunningTime="2026-05-28 21:02:40.435141511 +0000 UTC m=+610.552535109" watchObservedRunningTime="2026-05-28 21:02:40.436399072 +0000 UTC m=+610.553792658" May 28 21:02:40.459848 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:40.459823 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2lt72"] May 28 21:02:42.417051 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:42.417010 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-2lt72" podUID="99414f6e-cc6a-4745-a6e4-eeb3c6995d16" containerName="authorino" containerID="cri-o://938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd" gracePeriod=30 May 28 21:02:42.659261 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:42.659228 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-2lt72" May 28 21:02:42.728710 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:42.728679 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8jqh\" (UniqueName: \"kubernetes.io/projected/99414f6e-cc6a-4745-a6e4-eeb3c6995d16-kube-api-access-l8jqh\") pod \"99414f6e-cc6a-4745-a6e4-eeb3c6995d16\" (UID: \"99414f6e-cc6a-4745-a6e4-eeb3c6995d16\") " May 28 21:02:42.730735 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:42.730703 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99414f6e-cc6a-4745-a6e4-eeb3c6995d16-kube-api-access-l8jqh" (OuterVolumeSpecName: "kube-api-access-l8jqh") pod "99414f6e-cc6a-4745-a6e4-eeb3c6995d16" (UID: "99414f6e-cc6a-4745-a6e4-eeb3c6995d16"). InnerVolumeSpecName "kube-api-access-l8jqh". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 21:02:42.829466 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:42.829423 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8jqh\" (UniqueName: \"kubernetes.io/projected/99414f6e-cc6a-4745-a6e4-eeb3c6995d16-kube-api-access-l8jqh\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 21:02:43.421540 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:43.421509 2563 generic.go:358] "Generic (PLEG): container finished" podID="99414f6e-cc6a-4745-a6e4-eeb3c6995d16" containerID="938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd" exitCode=0 May 28 21:02:43.421983 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:43.421548 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-2lt72" event={"ID":"99414f6e-cc6a-4745-a6e4-eeb3c6995d16","Type":"ContainerDied","Data":"938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd"} May 28 21:02:43.421983 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:43.421558 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-2lt72" May 28 21:02:43.421983 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:43.421573 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-2lt72" event={"ID":"99414f6e-cc6a-4745-a6e4-eeb3c6995d16","Type":"ContainerDied","Data":"f2b067fee061098a8abba079a8871151935d121c9f0826aeb98bc41cae4e2eb0"} May 28 21:02:43.421983 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:43.421589 2563 scope.go:117] "RemoveContainer" containerID="938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd" May 28 21:02:43.429758 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:43.429741 2563 scope.go:117] "RemoveContainer" containerID="938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd" May 28 21:02:43.429998 ip-10-0-140-94 kubenswrapper[2563]: E0528 21:02:43.429974 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd\": container with ID starting with 938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd not found: ID does not exist" containerID="938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd" May 28 21:02:43.430062 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:43.430006 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd"} err="failed to get container status \"938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd\": rpc error: code = NotFound desc = could not find container \"938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd\": container with ID starting with 938072723a1103dc311c13326ae1e1e5d8700bfa489b1116a10bd94f26bcbedd not found: ID does not exist" May 28 21:02:43.440593 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:43.440569 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2lt72"] May 28 21:02:43.444085 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:43.444064 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2lt72"] May 28 21:02:44.467569 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:02:44.467539 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99414f6e-cc6a-4745-a6e4-eeb3c6995d16" path="/var/lib/kubelet/pods/99414f6e-cc6a-4745-a6e4-eeb3c6995d16/volumes" May 28 21:03:27.209271 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.209234 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] May 28 21:03:27.209674 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.209594 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99414f6e-cc6a-4745-a6e4-eeb3c6995d16" containerName="authorino" May 28 21:03:27.209674 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.209607 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="99414f6e-cc6a-4745-a6e4-eeb3c6995d16" containerName="authorino" May 28 21:03:27.209674 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.209668 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="99414f6e-cc6a-4745-a6e4-eeb3c6995d16" containerName="authorino" May 28 21:03:27.211427 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.211412 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" May 28 21:03:27.213474 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.213449 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-z4ttf\"" May 28 21:03:27.213597 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.213453 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" May 28 21:03:27.213597 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.213499 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" May 28 21:03:27.214039 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.214022 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" May 28 21:03:27.220571 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.220546 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] May 28 21:03:27.307618 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.307587 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6hml\" (UniqueName: \"kubernetes.io/projected/d08ddde9-bfdb-460f-a391-e5f095759612-kube-api-access-x6hml\") pod \"maas-keycloak-0\" (UID: \"d08ddde9-bfdb-460f-a391-e5f095759612\") " pod="keycloak-system/maas-keycloak-0" May 28 21:03:27.408595 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.408563 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hml\" (UniqueName: \"kubernetes.io/projected/d08ddde9-bfdb-460f-a391-e5f095759612-kube-api-access-x6hml\") pod \"maas-keycloak-0\" (UID: \"d08ddde9-bfdb-460f-a391-e5f095759612\") " pod="keycloak-system/maas-keycloak-0" May 28 21:03:27.416187 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.416149 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6hml\" (UniqueName: \"kubernetes.io/projected/d08ddde9-bfdb-460f-a391-e5f095759612-kube-api-access-x6hml\") pod \"maas-keycloak-0\" (UID: \"d08ddde9-bfdb-460f-a391-e5f095759612\") " pod="keycloak-system/maas-keycloak-0" May 28 21:03:27.521722 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.521695 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" May 28 21:03:27.647639 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:27.647600 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] May 28 21:03:27.649788 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:03:27.649763 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08ddde9_bfdb_460f_a391_e5f095759612.slice/crio-94b04f47230cd9d3db69d7c1cbf8a252fe9e7995b911f49371fa5c0a2cc8af94 WatchSource:0}: Error finding container 94b04f47230cd9d3db69d7c1cbf8a252fe9e7995b911f49371fa5c0a2cc8af94: Status 404 returned error can't find the container with id 94b04f47230cd9d3db69d7c1cbf8a252fe9e7995b911f49371fa5c0a2cc8af94 May 28 21:03:28.570256 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:28.570212 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d08ddde9-bfdb-460f-a391-e5f095759612","Type":"ContainerStarted","Data":"94b04f47230cd9d3db69d7c1cbf8a252fe9e7995b911f49371fa5c0a2cc8af94"} May 28 21:03:32.049250 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:32.049228 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" May 28 21:03:32.588248 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:32.588207 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d08ddde9-bfdb-460f-a391-e5f095759612","Type":"ContainerStarted","Data":"6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a"} May 28 21:03:32.607828 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:32.607769 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.212186951 podStartE2EDuration="5.607752645s" podCreationTimestamp="2026-05-28 21:03:27 +0000 UTC" firstStartedPulling="2026-05-28 21:03:27.651093802 +0000 UTC m=+657.768487364" lastFinishedPulling="2026-05-28 21:03:32.046659493 +0000 UTC m=+662.164053058" observedRunningTime="2026-05-28 21:03:32.605081 +0000 UTC m=+662.722474583" watchObservedRunningTime="2026-05-28 21:03:32.607752645 +0000 UTC m=+662.725146231" May 28 21:03:33.522216 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:33.522132 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" May 28 21:03:33.523833 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:33.523783 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:34.522264 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:34.522213 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:35.523021 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:35.522972 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:36.522603 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:36.522555 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:37.522798 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:37.522758 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" May 28 21:03:37.524081 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:37.524049 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:38.522481 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:38.522419 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:39.522532 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:39.522473 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:40.522333 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:40.522281 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:41.523225 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:41.523150 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:42.522809 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:42.522767 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:43.522258 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:43.522214 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:44.522613 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:44.522564 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" May 28 21:03:45.614405 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:45.614364 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" May 28 21:03:45.629087 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:45.629042 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 28 21:03:55.620928 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:55.620895 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" May 28 21:03:56.489965 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.489935 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-pv7vx"] May 28 21:03:56.492621 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.492604 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-pv7vx" May 28 21:03:56.497455 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.497432 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-pv7vx"] May 28 21:03:56.586062 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.586035 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtxqg\" (UniqueName: \"kubernetes.io/projected/db9539bf-034c-4754-94e7-edcb9a923911-kube-api-access-gtxqg\") pod \"authorino-8b475cf9f-pv7vx\" (UID: \"db9539bf-034c-4754-94e7-edcb9a923911\") " pod="kuadrant-system/authorino-8b475cf9f-pv7vx" May 28 21:03:56.686947 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.686914 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gtxqg\" (UniqueName: \"kubernetes.io/projected/db9539bf-034c-4754-94e7-edcb9a923911-kube-api-access-gtxqg\") pod \"authorino-8b475cf9f-pv7vx\" (UID: \"db9539bf-034c-4754-94e7-edcb9a923911\") " pod="kuadrant-system/authorino-8b475cf9f-pv7vx" May 28 21:03:56.694448 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.694425 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtxqg\" (UniqueName: \"kubernetes.io/projected/db9539bf-034c-4754-94e7-edcb9a923911-kube-api-access-gtxqg\") pod \"authorino-8b475cf9f-pv7vx\" (UID: \"db9539bf-034c-4754-94e7-edcb9a923911\") " pod="kuadrant-system/authorino-8b475cf9f-pv7vx" May 28 21:03:56.719217 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.719189 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-pv7vx"] May 28 21:03:56.719383 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.719373 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-pv7vx" May 28 21:03:56.743294 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.743235 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59f94b4fdd-ln8nb"] May 28 21:03:56.750543 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.748447 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" May 28 21:03:56.755711 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.755689 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59f94b4fdd-ln8nb"] May 28 21:03:56.848728 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.848696 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-pv7vx"] May 28 21:03:56.851914 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:03:56.851886 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb9539bf_034c_4754_94e7_edcb9a923911.slice/crio-6a934cb6b4c5709477c24f6065fd6ca9aaed9cf146aa25b7c1fd8dfd69f4704b WatchSource:0}: Error finding container 6a934cb6b4c5709477c24f6065fd6ca9aaed9cf146aa25b7c1fd8dfd69f4704b: Status 404 returned error can't find the container with id 6a934cb6b4c5709477c24f6065fd6ca9aaed9cf146aa25b7c1fd8dfd69f4704b May 28 21:03:56.887820 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.887787 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8x7c\" (UniqueName: \"kubernetes.io/projected/dba7f850-11ce-4a6c-99c9-bd3a46b02eeb-kube-api-access-t8x7c\") pod \"authorino-59f94b4fdd-ln8nb\" (UID: \"dba7f850-11ce-4a6c-99c9-bd3a46b02eeb\") " pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" May 28 21:03:56.988730 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.988704 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t8x7c\" (UniqueName: \"kubernetes.io/projected/dba7f850-11ce-4a6c-99c9-bd3a46b02eeb-kube-api-access-t8x7c\") pod \"authorino-59f94b4fdd-ln8nb\" (UID: \"dba7f850-11ce-4a6c-99c9-bd3a46b02eeb\") " pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" May 28 21:03:56.996888 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:56.996837 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8x7c\" (UniqueName: \"kubernetes.io/projected/dba7f850-11ce-4a6c-99c9-bd3a46b02eeb-kube-api-access-t8x7c\") pod \"authorino-59f94b4fdd-ln8nb\" (UID: \"dba7f850-11ce-4a6c-99c9-bd3a46b02eeb\") " pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" May 28 21:03:57.062038 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.061999 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" May 28 21:03:57.186207 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.186151 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59f94b4fdd-ln8nb"] May 28 21:03:57.189661 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:03:57.189632 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddba7f850_11ce_4a6c_99c9_bd3a46b02eeb.slice/crio-8a6cbdff915422a4c36e5213f500d527f33c6419bd3ac08b4a7e1714c54ca9d1 WatchSource:0}: Error finding container 8a6cbdff915422a4c36e5213f500d527f33c6419bd3ac08b4a7e1714c54ca9d1: Status 404 returned error can't find the container with id 8a6cbdff915422a4c36e5213f500d527f33c6419bd3ac08b4a7e1714c54ca9d1 May 28 21:03:57.479607 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.479575 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59f94b4fdd-ln8nb"] May 28 21:03:57.505948 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.505921 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78f59b79fb-x8mr4"] May 28 21:03:57.508036 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.508018 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f59b79fb-x8mr4" May 28 21:03:57.509866 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.509843 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" May 28 21:03:57.517364 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.517345 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78f59b79fb-x8mr4"] May 28 21:03:57.594151 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.594124 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjtdr\" (UniqueName: \"kubernetes.io/projected/61aae3c9-527e-430e-ad5f-62fe897cd6b1-kube-api-access-vjtdr\") pod \"authorino-78f59b79fb-x8mr4\" (UID: \"61aae3c9-527e-430e-ad5f-62fe897cd6b1\") " pod="kuadrant-system/authorino-78f59b79fb-x8mr4" May 28 21:03:57.594260 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.594207 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61aae3c9-527e-430e-ad5f-62fe897cd6b1-tls-cert\") pod \"authorino-78f59b79fb-x8mr4\" (UID: \"61aae3c9-527e-430e-ad5f-62fe897cd6b1\") " pod="kuadrant-system/authorino-78f59b79fb-x8mr4" May 28 21:03:57.693389 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.693355 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-pv7vx" event={"ID":"db9539bf-034c-4754-94e7-edcb9a923911","Type":"ContainerStarted","Data":"0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd"} May 28 21:03:57.693389 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.693383 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-pv7vx" podUID="db9539bf-034c-4754-94e7-edcb9a923911" containerName="authorino" containerID="cri-o://0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd" gracePeriod=30 May 28 21:03:57.693912 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.693395 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-pv7vx" event={"ID":"db9539bf-034c-4754-94e7-edcb9a923911","Type":"ContainerStarted","Data":"6a934cb6b4c5709477c24f6065fd6ca9aaed9cf146aa25b7c1fd8dfd69f4704b"} May 28 21:03:57.694705 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.694679 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vjtdr\" (UniqueName: \"kubernetes.io/projected/61aae3c9-527e-430e-ad5f-62fe897cd6b1-kube-api-access-vjtdr\") pod \"authorino-78f59b79fb-x8mr4\" (UID: \"61aae3c9-527e-430e-ad5f-62fe897cd6b1\") " pod="kuadrant-system/authorino-78f59b79fb-x8mr4" May 28 21:03:57.694851 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.694742 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61aae3c9-527e-430e-ad5f-62fe897cd6b1-tls-cert\") pod \"authorino-78f59b79fb-x8mr4\" (UID: \"61aae3c9-527e-430e-ad5f-62fe897cd6b1\") " pod="kuadrant-system/authorino-78f59b79fb-x8mr4" May 28 21:03:57.694851 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.694773 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" event={"ID":"dba7f850-11ce-4a6c-99c9-bd3a46b02eeb","Type":"ContainerStarted","Data":"4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925"} May 28 21:03:57.694851 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.694801 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" event={"ID":"dba7f850-11ce-4a6c-99c9-bd3a46b02eeb","Type":"ContainerStarted","Data":"8a6cbdff915422a4c36e5213f500d527f33c6419bd3ac08b4a7e1714c54ca9d1"} May 28 21:03:57.694974 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.694897 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" podUID="dba7f850-11ce-4a6c-99c9-bd3a46b02eeb" containerName="authorino" containerID="cri-o://4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925" gracePeriod=30 May 28 21:03:57.697642 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.697596 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61aae3c9-527e-430e-ad5f-62fe897cd6b1-tls-cert\") pod \"authorino-78f59b79fb-x8mr4\" (UID: \"61aae3c9-527e-430e-ad5f-62fe897cd6b1\") " pod="kuadrant-system/authorino-78f59b79fb-x8mr4" May 28 21:03:57.704058 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.704037 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjtdr\" (UniqueName: \"kubernetes.io/projected/61aae3c9-527e-430e-ad5f-62fe897cd6b1-kube-api-access-vjtdr\") pod \"authorino-78f59b79fb-x8mr4\" (UID: \"61aae3c9-527e-430e-ad5f-62fe897cd6b1\") " pod="kuadrant-system/authorino-78f59b79fb-x8mr4" May 28 21:03:57.706465 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.706422 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-pv7vx" podStartSLOduration=1.364100065 podStartE2EDuration="1.706407104s" podCreationTimestamp="2026-05-28 21:03:56 +0000 UTC" firstStartedPulling="2026-05-28 21:03:56.853476774 +0000 UTC m=+686.970870353" lastFinishedPulling="2026-05-28 21:03:57.19578383 +0000 UTC m=+687.313177392" observedRunningTime="2026-05-28 21:03:57.705697816 +0000 UTC m=+687.823091401" watchObservedRunningTime="2026-05-28 21:03:57.706407104 +0000 UTC m=+687.823800690" May 28 21:03:57.718826 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.718780 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" podStartSLOduration=1.364052822 podStartE2EDuration="1.718761471s" podCreationTimestamp="2026-05-28 21:03:56 +0000 UTC" firstStartedPulling="2026-05-28 21:03:57.190978878 +0000 UTC m=+687.308372440" lastFinishedPulling="2026-05-28 21:03:57.545687527 +0000 UTC m=+687.663081089" observedRunningTime="2026-05-28 21:03:57.717722497 +0000 UTC m=+687.835116083" watchObservedRunningTime="2026-05-28 21:03:57.718761471 +0000 UTC m=+687.836155055" May 28 21:03:57.817276 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.817255 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f59b79fb-x8mr4" May 28 21:03:57.963310 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.963277 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78f59b79fb-x8mr4"] May 28 21:03:57.965556 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:03:57.965519 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61aae3c9_527e_430e_ad5f_62fe897cd6b1.slice/crio-5e158377871656ab30bc12ed91dae8b87ada013d709e00d6e72ace7c3c8da89d WatchSource:0}: Error finding container 5e158377871656ab30bc12ed91dae8b87ada013d709e00d6e72ace7c3c8da89d: Status 404 returned error can't find the container with id 5e158377871656ab30bc12ed91dae8b87ada013d709e00d6e72ace7c3c8da89d May 28 21:03:57.966795 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.966779 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-pv7vx" May 28 21:03:57.970194 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:57.970163 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" May 28 21:03:58.097929 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.097855 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8x7c\" (UniqueName: \"kubernetes.io/projected/dba7f850-11ce-4a6c-99c9-bd3a46b02eeb-kube-api-access-t8x7c\") pod \"dba7f850-11ce-4a6c-99c9-bd3a46b02eeb\" (UID: \"dba7f850-11ce-4a6c-99c9-bd3a46b02eeb\") " May 28 21:03:58.097929 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.097919 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtxqg\" (UniqueName: \"kubernetes.io/projected/db9539bf-034c-4754-94e7-edcb9a923911-kube-api-access-gtxqg\") pod \"db9539bf-034c-4754-94e7-edcb9a923911\" (UID: \"db9539bf-034c-4754-94e7-edcb9a923911\") " May 28 21:03:58.099962 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.099934 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba7f850-11ce-4a6c-99c9-bd3a46b02eeb-kube-api-access-t8x7c" (OuterVolumeSpecName: "kube-api-access-t8x7c") pod "dba7f850-11ce-4a6c-99c9-bd3a46b02eeb" (UID: "dba7f850-11ce-4a6c-99c9-bd3a46b02eeb"). InnerVolumeSpecName "kube-api-access-t8x7c". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 21:03:58.100081 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.099978 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db9539bf-034c-4754-94e7-edcb9a923911-kube-api-access-gtxqg" (OuterVolumeSpecName: "kube-api-access-gtxqg") pod "db9539bf-034c-4754-94e7-edcb9a923911" (UID: "db9539bf-034c-4754-94e7-edcb9a923911"). InnerVolumeSpecName "kube-api-access-gtxqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 21:03:58.199442 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.199415 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8x7c\" (UniqueName: \"kubernetes.io/projected/dba7f850-11ce-4a6c-99c9-bd3a46b02eeb-kube-api-access-t8x7c\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 21:03:58.199442 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.199437 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtxqg\" (UniqueName: \"kubernetes.io/projected/db9539bf-034c-4754-94e7-edcb9a923911-kube-api-access-gtxqg\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 21:03:58.699143 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.699107 2563 generic.go:358] "Generic (PLEG): container finished" podID="dba7f850-11ce-4a6c-99c9-bd3a46b02eeb" containerID="4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925" exitCode=0 May 28 21:03:58.699601 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.699156 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" May 28 21:03:58.699601 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.699202 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" event={"ID":"dba7f850-11ce-4a6c-99c9-bd3a46b02eeb","Type":"ContainerDied","Data":"4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925"} May 28 21:03:58.699601 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.699252 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59f94b4fdd-ln8nb" event={"ID":"dba7f850-11ce-4a6c-99c9-bd3a46b02eeb","Type":"ContainerDied","Data":"8a6cbdff915422a4c36e5213f500d527f33c6419bd3ac08b4a7e1714c54ca9d1"} May 28 21:03:58.699601 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.699276 2563 scope.go:117] "RemoveContainer" containerID="4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925" May 28 21:03:58.700405 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.700386 2563 generic.go:358] "Generic (PLEG): container finished" podID="db9539bf-034c-4754-94e7-edcb9a923911" containerID="0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd" exitCode=0 May 28 21:03:58.700484 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.700432 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-pv7vx" May 28 21:03:58.700484 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.700452 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-pv7vx" event={"ID":"db9539bf-034c-4754-94e7-edcb9a923911","Type":"ContainerDied","Data":"0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd"} May 28 21:03:58.700484 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.700476 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-pv7vx" event={"ID":"db9539bf-034c-4754-94e7-edcb9a923911","Type":"ContainerDied","Data":"6a934cb6b4c5709477c24f6065fd6ca9aaed9cf146aa25b7c1fd8dfd69f4704b"} May 28 21:03:58.702014 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.701948 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f59b79fb-x8mr4" event={"ID":"61aae3c9-527e-430e-ad5f-62fe897cd6b1","Type":"ContainerStarted","Data":"aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871"} May 28 21:03:58.702014 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.701981 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f59b79fb-x8mr4" event={"ID":"61aae3c9-527e-430e-ad5f-62fe897cd6b1","Type":"ContainerStarted","Data":"5e158377871656ab30bc12ed91dae8b87ada013d709e00d6e72ace7c3c8da89d"} May 28 21:03:58.714636 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.714616 2563 scope.go:117] "RemoveContainer" containerID="4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925" May 28 21:03:58.714926 ip-10-0-140-94 kubenswrapper[2563]: E0528 21:03:58.714900 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925\": container with ID starting with 4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925 not found: ID does not exist" containerID="4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925" May 28 21:03:58.714996 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.714936 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925"} err="failed to get container status \"4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925\": rpc error: code = NotFound desc = could not find container \"4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925\": container with ID starting with 4ff83540a1d5b9e5e60e8445c3bc0e5f9d4b759ba60d88762bd7d8149af0a925 not found: ID does not exist" May 28 21:03:58.714996 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.714958 2563 scope.go:117] "RemoveContainer" containerID="0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd" May 28 21:03:58.716538 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.716485 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-78f59b79fb-x8mr4" podStartSLOduration=1.4247540029999999 podStartE2EDuration="1.716470528s" podCreationTimestamp="2026-05-28 21:03:57 +0000 UTC" firstStartedPulling="2026-05-28 21:03:57.96694327 +0000 UTC m=+688.084336837" lastFinishedPulling="2026-05-28 21:03:58.258659791 +0000 UTC m=+688.376053362" observedRunningTime="2026-05-28 21:03:58.714875696 +0000 UTC m=+688.832269281" watchObservedRunningTime="2026-05-28 21:03:58.716470528 +0000 UTC m=+688.833864115" May 28 21:03:58.722032 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.722006 2563 scope.go:117] "RemoveContainer" containerID="0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd" May 28 21:03:58.722307 ip-10-0-140-94 kubenswrapper[2563]: E0528 21:03:58.722290 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd\": container with ID starting with 0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd not found: ID does not exist" containerID="0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd" May 28 21:03:58.722355 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.722311 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd"} err="failed to get container status \"0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd\": rpc error: code = NotFound desc = could not find container \"0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd\": container with ID starting with 0bc1baa26cfba12e2a353c91f163151f6f5ddddcf5ee1f7244d962d77b0050cd not found: ID does not exist" May 28 21:03:58.734200 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.734153 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59f94b4fdd-ln8nb"] May 28 21:03:58.739573 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.739552 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59f94b4fdd-ln8nb"] May 28 21:03:58.743002 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.742982 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-nd5cj"] May 28 21:03:58.743197 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.743157 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-nd5cj" podUID="3492266a-82d8-4c8b-9b49-da9a98735f6e" containerName="authorino" containerID="cri-o://313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8" gracePeriod=30 May 28 21:03:58.757542 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.757517 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-pv7vx"] May 28 21:03:58.760680 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.760656 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-pv7vx"] May 28 21:03:58.987209 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:58.987133 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-nd5cj" May 28 21:03:59.123948 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.123916 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrdp7\" (UniqueName: \"kubernetes.io/projected/3492266a-82d8-4c8b-9b49-da9a98735f6e-kube-api-access-lrdp7\") pod \"3492266a-82d8-4c8b-9b49-da9a98735f6e\" (UID: \"3492266a-82d8-4c8b-9b49-da9a98735f6e\") " May 28 21:03:59.125927 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.125898 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3492266a-82d8-4c8b-9b49-da9a98735f6e-kube-api-access-lrdp7" (OuterVolumeSpecName: "kube-api-access-lrdp7") pod "3492266a-82d8-4c8b-9b49-da9a98735f6e" (UID: "3492266a-82d8-4c8b-9b49-da9a98735f6e"). InnerVolumeSpecName "kube-api-access-lrdp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 21:03:59.224709 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.224683 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrdp7\" (UniqueName: \"kubernetes.io/projected/3492266a-82d8-4c8b-9b49-da9a98735f6e-kube-api-access-lrdp7\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 21:03:59.707782 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.707744 2563 generic.go:358] "Generic (PLEG): container finished" podID="3492266a-82d8-4c8b-9b49-da9a98735f6e" containerID="313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8" exitCode=0 May 28 21:03:59.708254 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.707795 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-nd5cj" May 28 21:03:59.708254 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.707823 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-nd5cj" event={"ID":"3492266a-82d8-4c8b-9b49-da9a98735f6e","Type":"ContainerDied","Data":"313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8"} May 28 21:03:59.708254 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.707858 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-nd5cj" event={"ID":"3492266a-82d8-4c8b-9b49-da9a98735f6e","Type":"ContainerDied","Data":"68ae751f1060723eece96c93fdea8d9b0542238426ac4c68c3b0f58006e43f17"} May 28 21:03:59.708254 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.707874 2563 scope.go:117] "RemoveContainer" containerID="313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8" May 28 21:03:59.716245 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.716230 2563 scope.go:117] "RemoveContainer" containerID="313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8" May 28 21:03:59.716454 ip-10-0-140-94 kubenswrapper[2563]: E0528 21:03:59.716436 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8\": container with ID starting with 313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8 not found: ID does not exist" containerID="313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8" May 28 21:03:59.716503 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.716462 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8"} err="failed to get container status \"313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8\": rpc error: code = NotFound desc = could not find container \"313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8\": container with ID starting with 313b94e84738ce2fc42c9d6af9d56cfc3b02f473f625d919b423aff8afc6a6a8 not found: ID does not exist" May 28 21:03:59.727347 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.727314 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-nd5cj"] May 28 21:03:59.731221 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:03:59.731202 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-nd5cj"] May 28 21:04:00.467834 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:00.467801 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3492266a-82d8-4c8b-9b49-da9a98735f6e" path="/var/lib/kubelet/pods/3492266a-82d8-4c8b-9b49-da9a98735f6e/volumes" May 28 21:04:00.468118 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:00.468106 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db9539bf-034c-4754-94e7-edcb9a923911" path="/var/lib/kubelet/pods/db9539bf-034c-4754-94e7-edcb9a923911/volumes" May 28 21:04:00.468410 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:00.468398 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dba7f850-11ce-4a6c-99c9-bd3a46b02eeb" path="/var/lib/kubelet/pods/dba7f850-11ce-4a6c-99c9-bd3a46b02eeb/volumes" May 28 21:04:43.903439 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:43.903383 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] May 28 21:04:43.903941 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:43.903612 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" containerID="cri-o://6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a" gracePeriod=30 May 28 21:04:45.546051 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.546029 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" May 28 21:04:45.695699 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.695625 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6hml\" (UniqueName: \"kubernetes.io/projected/d08ddde9-bfdb-460f-a391-e5f095759612-kube-api-access-x6hml\") pod \"d08ddde9-bfdb-460f-a391-e5f095759612\" (UID: \"d08ddde9-bfdb-460f-a391-e5f095759612\") " May 28 21:04:45.697661 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.697630 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08ddde9-bfdb-460f-a391-e5f095759612-kube-api-access-x6hml" (OuterVolumeSpecName: "kube-api-access-x6hml") pod "d08ddde9-bfdb-460f-a391-e5f095759612" (UID: "d08ddde9-bfdb-460f-a391-e5f095759612"). InnerVolumeSpecName "kube-api-access-x6hml". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 21:04:45.796618 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.796573 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6hml\" (UniqueName: \"kubernetes.io/projected/d08ddde9-bfdb-460f-a391-e5f095759612-kube-api-access-x6hml\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 21:04:45.863216 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.863149 2563 generic.go:358] "Generic (PLEG): container finished" podID="d08ddde9-bfdb-460f-a391-e5f095759612" containerID="6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a" exitCode=143 May 28 21:04:45.863375 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.863240 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" May 28 21:04:45.863375 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.863246 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d08ddde9-bfdb-460f-a391-e5f095759612","Type":"ContainerDied","Data":"6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a"} May 28 21:04:45.863375 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.863281 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d08ddde9-bfdb-460f-a391-e5f095759612","Type":"ContainerDied","Data":"94b04f47230cd9d3db69d7c1cbf8a252fe9e7995b911f49371fa5c0a2cc8af94"} May 28 21:04:45.863375 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.863296 2563 scope.go:117] "RemoveContainer" containerID="6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a" May 28 21:04:45.872378 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.872363 2563 scope.go:117] "RemoveContainer" containerID="6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a" May 28 21:04:45.872629 ip-10-0-140-94 kubenswrapper[2563]: E0528 21:04:45.872611 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a\": container with ID starting with 6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a not found: ID does not exist" containerID="6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a" May 28 21:04:45.872678 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.872637 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a"} err="failed to get container status \"6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a\": rpc error: code = NotFound desc = could not find container \"6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a\": container with ID starting with 6c9bd070f2821b9d77d3c0a821a11b9c1abe23c9c8b2b793303df7691aa6370a not found: ID does not exist" May 28 21:04:45.883885 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.883862 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] May 28 21:04:45.887461 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.887438 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] May 28 21:04:45.909726 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.909704 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] May 28 21:04:45.910042 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910029 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" May 28 21:04:45.910086 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910043 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" May 28 21:04:45.910086 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910052 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db9539bf-034c-4754-94e7-edcb9a923911" containerName="authorino" May 28 21:04:45.910086 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910057 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="db9539bf-034c-4754-94e7-edcb9a923911" containerName="authorino" May 28 21:04:45.910086 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910068 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba7f850-11ce-4a6c-99c9-bd3a46b02eeb" containerName="authorino" May 28 21:04:45.910086 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910073 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba7f850-11ce-4a6c-99c9-bd3a46b02eeb" containerName="authorino" May 28 21:04:45.910086 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910088 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3492266a-82d8-4c8b-9b49-da9a98735f6e" containerName="authorino" May 28 21:04:45.910301 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910093 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3492266a-82d8-4c8b-9b49-da9a98735f6e" containerName="authorino" May 28 21:04:45.910301 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910141 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="dba7f850-11ce-4a6c-99c9-bd3a46b02eeb" containerName="authorino" May 28 21:04:45.910301 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910150 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="3492266a-82d8-4c8b-9b49-da9a98735f6e" containerName="authorino" May 28 21:04:45.910301 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910157 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" containerName="keycloak" May 28 21:04:45.910301 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.910179 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="db9539bf-034c-4754-94e7-edcb9a923911" containerName="authorino" May 28 21:04:45.912543 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.912528 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" May 28 21:04:45.914562 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.914539 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" May 28 21:04:45.915238 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.915209 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-z4ttf\"" May 28 21:04:45.915238 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.915209 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" May 28 21:04:45.915401 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.915357 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" May 28 21:04:45.915503 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.915480 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" May 28 21:04:45.923278 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.923254 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] May 28 21:04:45.997754 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.997725 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/4f1ee830-308d-4e9f-8bef-52b981665c92-test-realms\") pod \"maas-keycloak-0\" (UID: \"4f1ee830-308d-4e9f-8bef-52b981665c92\") " pod="keycloak-system/maas-keycloak-0" May 28 21:04:45.997901 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:45.997772 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mldp2\" (UniqueName: \"kubernetes.io/projected/4f1ee830-308d-4e9f-8bef-52b981665c92-kube-api-access-mldp2\") pod \"maas-keycloak-0\" (UID: \"4f1ee830-308d-4e9f-8bef-52b981665c92\") " pod="keycloak-system/maas-keycloak-0" May 28 21:04:46.098700 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:46.098668 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/4f1ee830-308d-4e9f-8bef-52b981665c92-test-realms\") pod \"maas-keycloak-0\" (UID: \"4f1ee830-308d-4e9f-8bef-52b981665c92\") " pod="keycloak-system/maas-keycloak-0" May 28 21:04:46.098823 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:46.098713 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mldp2\" (UniqueName: \"kubernetes.io/projected/4f1ee830-308d-4e9f-8bef-52b981665c92-kube-api-access-mldp2\") pod \"maas-keycloak-0\" (UID: \"4f1ee830-308d-4e9f-8bef-52b981665c92\") " pod="keycloak-system/maas-keycloak-0" May 28 21:04:46.099349 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:46.099330 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/4f1ee830-308d-4e9f-8bef-52b981665c92-test-realms\") pod \"maas-keycloak-0\" (UID: \"4f1ee830-308d-4e9f-8bef-52b981665c92\") " pod="keycloak-system/maas-keycloak-0" May 28 21:04:46.105867 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:46.105844 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mldp2\" (UniqueName: \"kubernetes.io/projected/4f1ee830-308d-4e9f-8bef-52b981665c92-kube-api-access-mldp2\") pod \"maas-keycloak-0\" (UID: \"4f1ee830-308d-4e9f-8bef-52b981665c92\") " pod="keycloak-system/maas-keycloak-0" May 28 21:04:46.227544 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:46.227503 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" May 28 21:04:46.352460 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:46.352437 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] May 28 21:04:46.355154 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:04:46.355129 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f1ee830_308d_4e9f_8bef_52b981665c92.slice/crio-37d199e9134347d72698e9357df35e8cfc2135e5def6734f7c5f6bf5a6a7f1e0 WatchSource:0}: Error finding container 37d199e9134347d72698e9357df35e8cfc2135e5def6734f7c5f6bf5a6a7f1e0: Status 404 returned error can't find the container with id 37d199e9134347d72698e9357df35e8cfc2135e5def6734f7c5f6bf5a6a7f1e0 May 28 21:04:46.356782 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:46.356764 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 21:04:46.467748 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:46.467720 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d08ddde9-bfdb-460f-a391-e5f095759612" path="/var/lib/kubelet/pods/d08ddde9-bfdb-460f-a391-e5f095759612/volumes" May 28 21:04:46.868436 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:46.868403 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"4f1ee830-308d-4e9f-8bef-52b981665c92","Type":"ContainerStarted","Data":"f12382ebc5aa30c7a772722ddc6e4a2d763e922e084d34f761bec6dcee18a4cf"} May 28 21:04:46.868877 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:46.868446 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"4f1ee830-308d-4e9f-8bef-52b981665c92","Type":"ContainerStarted","Data":"37d199e9134347d72698e9357df35e8cfc2135e5def6734f7c5f6bf5a6a7f1e0"} May 28 21:04:46.886451 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:46.886400 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.501708808 podStartE2EDuration="1.886386279s" podCreationTimestamp="2026-05-28 21:04:45 +0000 UTC" firstStartedPulling="2026-05-28 21:04:46.356883663 +0000 UTC m=+736.474277227" lastFinishedPulling="2026-05-28 21:04:46.741561136 +0000 UTC m=+736.858954698" observedRunningTime="2026-05-28 21:04:46.884391768 +0000 UTC m=+737.001785354" watchObservedRunningTime="2026-05-28 21:04:46.886386279 +0000 UTC m=+737.003780148" May 28 21:04:47.227647 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:47.227615 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" May 28 21:04:47.228845 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:47.228812 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:48.228644 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:48.228598 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:49.228442 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:49.228395 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:50.228731 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:50.228680 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:51.228719 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:51.228671 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:52.228301 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:52.228257 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:53.228769 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:53.228407 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:54.228552 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:54.228504 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:55.228361 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:55.228314 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:56.228484 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:56.228435 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" May 28 21:04:56.228980 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:56.228775 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:57.228841 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:57.228790 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:58.228744 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:58.228691 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:04:59.228852 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:04:59.228813 2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.34:9000/health/started\": dial tcp 10.133.0.34:9000: connect: connection refused" May 28 21:05:00.323011 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:00.322971 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" May 28 21:05:00.337854 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:00.337802 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="4f1ee830-308d-4e9f-8bef-52b981665c92" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 28 21:05:10.329500 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:10.329458 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" May 28 21:05:21.064892 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.064858 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-86b69d559b-l244z"] May 28 21:05:21.067539 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.067522 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:05:21.069519 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.069500 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" May 28 21:05:21.074989 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.074965 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86b69d559b-l244z"] May 28 21:05:21.140778 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.140735 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/783c2cd8-baf8-45d3-8b64-451447ef0fab-tls-cert\") pod \"authorino-86b69d559b-l244z\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:05:21.140959 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.140799 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/783c2cd8-baf8-45d3-8b64-451447ef0fab-oidc-ca\") pod \"authorino-86b69d559b-l244z\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:05:21.140959 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.140842 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkd6b\" (UniqueName: \"kubernetes.io/projected/783c2cd8-baf8-45d3-8b64-451447ef0fab-kube-api-access-nkd6b\") pod \"authorino-86b69d559b-l244z\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:05:21.241312 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.241271 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/783c2cd8-baf8-45d3-8b64-451447ef0fab-tls-cert\") pod \"authorino-86b69d559b-l244z\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:05:21.241503 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.241334 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/783c2cd8-baf8-45d3-8b64-451447ef0fab-oidc-ca\") pod \"authorino-86b69d559b-l244z\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:05:21.241503 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.241383 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nkd6b\" (UniqueName: \"kubernetes.io/projected/783c2cd8-baf8-45d3-8b64-451447ef0fab-kube-api-access-nkd6b\") pod \"authorino-86b69d559b-l244z\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:05:21.242016 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.241995 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/783c2cd8-baf8-45d3-8b64-451447ef0fab-oidc-ca\") pod \"authorino-86b69d559b-l244z\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:05:21.243674 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.243649 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/783c2cd8-baf8-45d3-8b64-451447ef0fab-tls-cert\") pod \"authorino-86b69d559b-l244z\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:05:21.248386 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.248365 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkd6b\" (UniqueName: \"kubernetes.io/projected/783c2cd8-baf8-45d3-8b64-451447ef0fab-kube-api-access-nkd6b\") pod \"authorino-86b69d559b-l244z\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:05:21.377706 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.377631 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:05:21.503830 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:21.503808 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86b69d559b-l244z"] May 28 21:05:21.506535 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:05:21.506505 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod783c2cd8_baf8_45d3_8b64_451447ef0fab.slice/crio-0d7d1fc7dc0f98e0b6f63aee9cb5bc4d34816539c336e6e1dcd463bcec0d1f0b WatchSource:0}: Error finding container 0d7d1fc7dc0f98e0b6f63aee9cb5bc4d34816539c336e6e1dcd463bcec0d1f0b: Status 404 returned error can't find the container with id 0d7d1fc7dc0f98e0b6f63aee9cb5bc4d34816539c336e6e1dcd463bcec0d1f0b May 28 21:05:22.002971 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.002939 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86b69d559b-l244z" event={"ID":"783c2cd8-baf8-45d3-8b64-451447ef0fab","Type":"ContainerStarted","Data":"30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775"} May 28 21:05:22.003084 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.002978 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86b69d559b-l244z" event={"ID":"783c2cd8-baf8-45d3-8b64-451447ef0fab","Type":"ContainerStarted","Data":"0d7d1fc7dc0f98e0b6f63aee9cb5bc4d34816539c336e6e1dcd463bcec0d1f0b"} May 28 21:05:22.020533 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.020485 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-86b69d559b-l244z" podStartSLOduration=0.604330968 podStartE2EDuration="1.020468669s" podCreationTimestamp="2026-05-28 21:05:21 +0000 UTC" firstStartedPulling="2026-05-28 21:05:21.507820487 +0000 UTC m=+771.625214049" lastFinishedPulling="2026-05-28 21:05:21.923958185 +0000 UTC m=+772.041351750" observedRunningTime="2026-05-28 21:05:22.018407359 +0000 UTC m=+772.135800945" watchObservedRunningTime="2026-05-28 21:05:22.020468669 +0000 UTC m=+772.137862302" May 28 21:05:22.049519 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.049487 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78f59b79fb-x8mr4"] May 28 21:05:22.049801 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.049746 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-78f59b79fb-x8mr4" podUID="61aae3c9-527e-430e-ad5f-62fe897cd6b1" containerName="authorino" containerID="cri-o://aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871" gracePeriod=30 May 28 21:05:22.313657 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.313627 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f59b79fb-x8mr4" May 28 21:05:22.452839 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.452804 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjtdr\" (UniqueName: \"kubernetes.io/projected/61aae3c9-527e-430e-ad5f-62fe897cd6b1-kube-api-access-vjtdr\") pod \"61aae3c9-527e-430e-ad5f-62fe897cd6b1\" (UID: \"61aae3c9-527e-430e-ad5f-62fe897cd6b1\") " May 28 21:05:22.452839 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.452850 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61aae3c9-527e-430e-ad5f-62fe897cd6b1-tls-cert\") pod \"61aae3c9-527e-430e-ad5f-62fe897cd6b1\" (UID: \"61aae3c9-527e-430e-ad5f-62fe897cd6b1\") " May 28 21:05:22.455593 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.455549 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61aae3c9-527e-430e-ad5f-62fe897cd6b1-kube-api-access-vjtdr" (OuterVolumeSpecName: "kube-api-access-vjtdr") pod "61aae3c9-527e-430e-ad5f-62fe897cd6b1" (UID: "61aae3c9-527e-430e-ad5f-62fe897cd6b1"). InnerVolumeSpecName "kube-api-access-vjtdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 21:05:22.464247 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.464162 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61aae3c9-527e-430e-ad5f-62fe897cd6b1-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "61aae3c9-527e-430e-ad5f-62fe897cd6b1" (UID: "61aae3c9-527e-430e-ad5f-62fe897cd6b1"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 21:05:22.554226 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.554186 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjtdr\" (UniqueName: \"kubernetes.io/projected/61aae3c9-527e-430e-ad5f-62fe897cd6b1-kube-api-access-vjtdr\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 21:05:22.554226 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:22.554218 2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61aae3c9-527e-430e-ad5f-62fe897cd6b1-tls-cert\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 21:05:23.007816 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:23.007780 2563 generic.go:358] "Generic (PLEG): container finished" podID="61aae3c9-527e-430e-ad5f-62fe897cd6b1" containerID="aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871" exitCode=0 May 28 21:05:23.008006 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:23.007831 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f59b79fb-x8mr4" May 28 21:05:23.008006 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:23.007867 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f59b79fb-x8mr4" event={"ID":"61aae3c9-527e-430e-ad5f-62fe897cd6b1","Type":"ContainerDied","Data":"aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871"} May 28 21:05:23.008006 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:23.007916 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f59b79fb-x8mr4" event={"ID":"61aae3c9-527e-430e-ad5f-62fe897cd6b1","Type":"ContainerDied","Data":"5e158377871656ab30bc12ed91dae8b87ada013d709e00d6e72ace7c3c8da89d"} May 28 21:05:23.008006 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:23.007937 2563 scope.go:117] "RemoveContainer" containerID="aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871" May 28 21:05:23.016029 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:23.016011 2563 scope.go:117] "RemoveContainer" containerID="aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871" May 28 21:05:23.016326 ip-10-0-140-94 kubenswrapper[2563]: E0528 21:05:23.016308 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871\": container with ID starting with aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871 not found: ID does not exist" containerID="aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871" May 28 21:05:23.016377 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:23.016334 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871"} err="failed to get container status \"aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871\": rpc error: code = NotFound desc = could not find container \"aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871\": container with ID starting with aea0ffc774fd2b8fdd88cff74872e3179e756c5d12450ceb3c474640470fe871 not found: ID does not exist" May 28 21:05:23.022262 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:23.022239 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78f59b79fb-x8mr4"] May 28 21:05:23.026358 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:23.026336 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-78f59b79fb-x8mr4"] May 28 21:05:24.474076 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:24.474037 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61aae3c9-527e-430e-ad5f-62fe897cd6b1" path="/var/lib/kubelet/pods/61aae3c9-527e-430e-ad5f-62fe897cd6b1/volumes" May 28 21:05:31.709125 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.709093 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2"] May 28 21:05:31.709537 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.709481 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61aae3c9-527e-430e-ad5f-62fe897cd6b1" containerName="authorino" May 28 21:05:31.709537 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.709494 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="61aae3c9-527e-430e-ad5f-62fe897cd6b1" containerName="authorino" May 28 21:05:31.709603 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.709560 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="61aae3c9-527e-430e-ad5f-62fe897cd6b1" containerName="authorino" May 28 21:05:31.711949 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.711933 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.714033 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.714006 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" May 28 21:05:31.714148 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.714043 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" May 28 21:05:31.714480 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.714464 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5qz76\"" May 28 21:05:31.714538 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.714516 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" May 28 21:05:31.720861 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.720835 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2"] May 28 21:05:31.837713 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.837675 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.837887 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.837763 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.837887 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.837796 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.837887 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.837844 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.837887 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.837865 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jsdq\" (UniqueName: \"kubernetes.io/projected/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-kube-api-access-5jsdq\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.838055 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.837908 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.939072 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.939030 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.939072 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.939081 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.939323 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.939139 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.939323 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.939188 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.939323 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.939236 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.939323 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.939260 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5jsdq\" (UniqueName: \"kubernetes.io/projected/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-kube-api-access-5jsdq\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.939552 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.939525 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.939674 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.939562 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.939674 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.939588 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.941454 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.941436 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.941717 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.941701 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:31.946411 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:31.946391 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jsdq\" (UniqueName: \"kubernetes.io/projected/ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c-kube-api-access-5jsdq\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2\" (UID: \"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:32.022567 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:32.022545 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:32.148293 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:32.148148 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2"] May 28 21:05:32.150840 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:05:32.150803 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4cf9e0_4853_4e3d_bedc_f5cb3d8acb9c.slice/crio-9594e4daabf93dcbf13daac006b05838cc37e57b73b306ef518d5b12e271e6ae WatchSource:0}: Error finding container 9594e4daabf93dcbf13daac006b05838cc37e57b73b306ef518d5b12e271e6ae: Status 404 returned error can't find the container with id 9594e4daabf93dcbf13daac006b05838cc37e57b73b306ef518d5b12e271e6ae May 28 21:05:33.046439 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:33.046395 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" event={"ID":"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c","Type":"ContainerStarted","Data":"9594e4daabf93dcbf13daac006b05838cc37e57b73b306ef518d5b12e271e6ae"} May 28 21:05:38.070483 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:38.070434 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" event={"ID":"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c","Type":"ContainerStarted","Data":"ee45f75f682e748accb6b02b10b689e4576d390fab1fba494a7b2fc0adc6c995"} May 28 21:05:43.089746 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:43.089710 2563 generic.go:358] "Generic (PLEG): container finished" podID="ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c" containerID="ee45f75f682e748accb6b02b10b689e4576d390fab1fba494a7b2fc0adc6c995" exitCode=0 May 28 21:05:43.090115 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:43.089785 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" event={"ID":"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c","Type":"ContainerDied","Data":"ee45f75f682e748accb6b02b10b689e4576d390fab1fba494a7b2fc0adc6c995"} May 28 21:05:45.098726 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:45.098688 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" event={"ID":"ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c","Type":"ContainerStarted","Data":"28dadaf1fdf88818944d2ac38bcdc5a170efdf67ce46484b4f931b05e90209ce"} May 28 21:05:45.099098 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:45.098924 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:45.119489 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:45.119437 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" podStartSLOduration=1.9834420019999999 podStartE2EDuration="14.119424558s" podCreationTimestamp="2026-05-28 21:05:31 +0000 UTC" firstStartedPulling="2026-05-28 21:05:32.152759182 +0000 UTC m=+782.270152746" lastFinishedPulling="2026-05-28 21:05:44.288741739 +0000 UTC m=+794.406135302" observedRunningTime="2026-05-28 21:05:45.117276091 +0000 UTC m=+795.234669677" watchObservedRunningTime="2026-05-28 21:05:45.119424558 +0000 UTC m=+795.236818143" May 28 21:05:56.115563 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.115528 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2" May 28 21:05:56.809186 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.809140 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj"] May 28 21:05:56.827630 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.827602 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj"] May 28 21:05:56.827808 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.827741 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.829960 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.829940 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" May 28 21:05:56.847409 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.847389 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.847502 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.847419 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.847502 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.847473 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvwkj\" (UniqueName: \"kubernetes.io/projected/511a4a15-b5eb-45c7-8271-a22f974fa52a-kube-api-access-kvwkj\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.847502 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.847491 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/511a4a15-b5eb-45c7-8271-a22f974fa52a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.847668 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.847518 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.847668 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.847592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.948676 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.948639 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.948676 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.948680 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.948934 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.948699 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.948934 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.948825 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.949013 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.948951 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvwkj\" (UniqueName: \"kubernetes.io/projected/511a4a15-b5eb-45c7-8271-a22f974fa52a-kube-api-access-kvwkj\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.949013 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.948987 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/511a4a15-b5eb-45c7-8271-a22f974fa52a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.949013 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.949000 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.949157 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.949074 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.949250 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.949233 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.950917 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.950896 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/511a4a15-b5eb-45c7-8271-a22f974fa52a-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.951408 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.951388 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/511a4a15-b5eb-45c7-8271-a22f974fa52a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:56.956924 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:56.956902 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvwkj\" (UniqueName: \"kubernetes.io/projected/511a4a15-b5eb-45c7-8271-a22f974fa52a-kube-api-access-kvwkj\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj\" (UID: \"511a4a15-b5eb-45c7-8271-a22f974fa52a\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:57.138387 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:57.138306 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:05:57.265327 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:57.265305 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj"] May 28 21:05:57.267918 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:05:57.267893 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511a4a15_b5eb_45c7_8271_a22f974fa52a.slice/crio-43ae8f0f9f7fe1b639ff955568114ffd6d2a906dcd63eded68ac09a7799bfa80 WatchSource:0}: Error finding container 43ae8f0f9f7fe1b639ff955568114ffd6d2a906dcd63eded68ac09a7799bfa80: Status 404 returned error can't find the container with id 43ae8f0f9f7fe1b639ff955568114ffd6d2a906dcd63eded68ac09a7799bfa80 May 28 21:05:58.141003 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:58.140964 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" event={"ID":"511a4a15-b5eb-45c7-8271-a22f974fa52a","Type":"ContainerStarted","Data":"d55ec6433aa4ff6b82b3b22a5da0d7ae2850b4f17b2abdae8cddb2030ab88141"} May 28 21:05:58.141003 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:05:58.141002 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" event={"ID":"511a4a15-b5eb-45c7-8271-a22f974fa52a","Type":"ContainerStarted","Data":"43ae8f0f9f7fe1b639ff955568114ffd6d2a906dcd63eded68ac09a7799bfa80"} May 28 21:06:03.159079 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:03.159044 2563 generic.go:358] "Generic (PLEG): container finished" podID="511a4a15-b5eb-45c7-8271-a22f974fa52a" containerID="d55ec6433aa4ff6b82b3b22a5da0d7ae2850b4f17b2abdae8cddb2030ab88141" exitCode=0 May 28 21:06:03.159461 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:03.159091 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" event={"ID":"511a4a15-b5eb-45c7-8271-a22f974fa52a","Type":"ContainerDied","Data":"d55ec6433aa4ff6b82b3b22a5da0d7ae2850b4f17b2abdae8cddb2030ab88141"} May 28 21:06:04.164021 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:04.163975 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" event={"ID":"511a4a15-b5eb-45c7-8271-a22f974fa52a","Type":"ContainerStarted","Data":"cb00a0e9f6aefe2b63462273161149504bd03fe2c225483e756c7ef75bd551ad"} May 28 21:06:04.164530 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:04.164242 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:06:04.183345 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:04.183285 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" podStartSLOduration=7.761528949 podStartE2EDuration="8.183266315s" podCreationTimestamp="2026-05-28 21:05:56 +0000 UTC" firstStartedPulling="2026-05-28 21:06:03.159710162 +0000 UTC m=+813.277103726" lastFinishedPulling="2026-05-28 21:06:03.581447529 +0000 UTC m=+813.698841092" observedRunningTime="2026-05-28 21:06:04.181864954 +0000 UTC m=+814.299258551" watchObservedRunningTime="2026-05-28 21:06:04.183266315 +0000 UTC m=+814.300659901" May 28 21:06:15.180298 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:15.180270 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj" May 28 21:06:35.207020 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.206986 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl"] May 28 21:06:35.210682 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.210664 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.212681 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.212656 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" May 28 21:06:35.219705 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.219682 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl"] May 28 21:06:35.378878 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.378818 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.379076 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.378912 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.379076 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.378942 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.379076 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.378976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.379076 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.379044 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.379262 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.379088 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6jg8\" (UniqueName: \"kubernetes.io/projected/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-kube-api-access-p6jg8\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.480055 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.480025 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.480213 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.480086 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.480213 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.480114 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.480213 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.480145 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.480332 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.480305 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.480380 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.480361 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p6jg8\" (UniqueName: \"kubernetes.io/projected/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-kube-api-access-p6jg8\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.480467 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.480446 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.480519 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.480501 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.480587 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.480540 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.482323 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.482296 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.482693 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.482677 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.487237 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.487215 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6jg8\" (UniqueName: \"kubernetes.io/projected/4dcddbcc-7ac7-45eb-a225-f2239d5fd28a-kube-api-access-p6jg8\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-rclhl\" (UID: \"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.522153 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.522132 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:35.644036 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:35.644005 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl"] May 28 21:06:35.647310 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:06:35.647281 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dcddbcc_7ac7_45eb_a225_f2239d5fd28a.slice/crio-1597a38a59ce98b1af865eb66e68c439f4c9e6c0e891006d4be3e8c95a0c559f WatchSource:0}: Error finding container 1597a38a59ce98b1af865eb66e68c439f4c9e6c0e891006d4be3e8c95a0c559f: Status 404 returned error can't find the container with id 1597a38a59ce98b1af865eb66e68c439f4c9e6c0e891006d4be3e8c95a0c559f May 28 21:06:36.275442 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:36.275404 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" event={"ID":"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a","Type":"ContainerStarted","Data":"59530c4b461244ecb2953b00eacc605839ca7be55b46d9337902781120963d16"} May 28 21:06:36.275820 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:36.275453 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" event={"ID":"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a","Type":"ContainerStarted","Data":"1597a38a59ce98b1af865eb66e68c439f4c9e6c0e891006d4be3e8c95a0c559f"} May 28 21:06:41.294226 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:41.294115 2563 generic.go:358] "Generic (PLEG): container finished" podID="4dcddbcc-7ac7-45eb-a225-f2239d5fd28a" containerID="59530c4b461244ecb2953b00eacc605839ca7be55b46d9337902781120963d16" exitCode=0 May 28 21:06:41.294226 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:41.294198 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" event={"ID":"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a","Type":"ContainerDied","Data":"59530c4b461244ecb2953b00eacc605839ca7be55b46d9337902781120963d16"} May 28 21:06:42.304184 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:42.304140 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" event={"ID":"4dcddbcc-7ac7-45eb-a225-f2239d5fd28a","Type":"ContainerStarted","Data":"2c0594259797eded5a72460e6c1d916335da574c02778eb2e4e353700424a9de"} May 28 21:06:42.304548 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:42.304378 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:06:42.320932 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:42.320888 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" podStartSLOduration=6.82395283 podStartE2EDuration="7.32087599s" podCreationTimestamp="2026-05-28 21:06:35 +0000 UTC" firstStartedPulling="2026-05-28 21:06:41.294824658 +0000 UTC m=+851.412218224" lastFinishedPulling="2026-05-28 21:06:41.791747821 +0000 UTC m=+851.909141384" observedRunningTime="2026-05-28 21:06:42.319157324 +0000 UTC m=+852.436550909" watchObservedRunningTime="2026-05-28 21:06:42.32087599 +0000 UTC m=+852.438269575" May 28 21:06:53.319689 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:06:53.319662 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-rclhl" May 28 21:07:28.650132 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.650057 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-bfbc85845-zqx4s"] May 28 21:07:28.653632 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.653616 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bfbc85845-zqx4s" May 28 21:07:28.660057 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.660032 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bfbc85845-zqx4s"] May 28 21:07:28.737702 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.737664 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/89a13f94-f69b-4d64-bc49-0008f0573a8e-oidc-ca\") pod \"authorino-bfbc85845-zqx4s\" (UID: \"89a13f94-f69b-4d64-bc49-0008f0573a8e\") " pod="kuadrant-system/authorino-bfbc85845-zqx4s" May 28 21:07:28.737816 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.737718 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89a13f94-f69b-4d64-bc49-0008f0573a8e-tls-cert\") pod \"authorino-bfbc85845-zqx4s\" (UID: \"89a13f94-f69b-4d64-bc49-0008f0573a8e\") " pod="kuadrant-system/authorino-bfbc85845-zqx4s" May 28 21:07:28.737816 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.737749 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrgtf\" (UniqueName: \"kubernetes.io/projected/89a13f94-f69b-4d64-bc49-0008f0573a8e-kube-api-access-qrgtf\") pod \"authorino-bfbc85845-zqx4s\" (UID: \"89a13f94-f69b-4d64-bc49-0008f0573a8e\") " pod="kuadrant-system/authorino-bfbc85845-zqx4s" May 28 21:07:28.838821 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.838774 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/89a13f94-f69b-4d64-bc49-0008f0573a8e-oidc-ca\") pod \"authorino-bfbc85845-zqx4s\" (UID: \"89a13f94-f69b-4d64-bc49-0008f0573a8e\") " pod="kuadrant-system/authorino-bfbc85845-zqx4s" May 28 21:07:28.839014 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.838925 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89a13f94-f69b-4d64-bc49-0008f0573a8e-tls-cert\") pod \"authorino-bfbc85845-zqx4s\" (UID: \"89a13f94-f69b-4d64-bc49-0008f0573a8e\") " pod="kuadrant-system/authorino-bfbc85845-zqx4s" May 28 21:07:28.839014 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.838971 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qrgtf\" (UniqueName: \"kubernetes.io/projected/89a13f94-f69b-4d64-bc49-0008f0573a8e-kube-api-access-qrgtf\") pod \"authorino-bfbc85845-zqx4s\" (UID: \"89a13f94-f69b-4d64-bc49-0008f0573a8e\") " pod="kuadrant-system/authorino-bfbc85845-zqx4s" May 28 21:07:28.839470 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.839449 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/89a13f94-f69b-4d64-bc49-0008f0573a8e-oidc-ca\") pod \"authorino-bfbc85845-zqx4s\" (UID: \"89a13f94-f69b-4d64-bc49-0008f0573a8e\") " pod="kuadrant-system/authorino-bfbc85845-zqx4s" May 28 21:07:28.841502 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.841482 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89a13f94-f69b-4d64-bc49-0008f0573a8e-tls-cert\") pod \"authorino-bfbc85845-zqx4s\" (UID: \"89a13f94-f69b-4d64-bc49-0008f0573a8e\") " pod="kuadrant-system/authorino-bfbc85845-zqx4s" May 28 21:07:28.852240 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.852220 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrgtf\" (UniqueName: \"kubernetes.io/projected/89a13f94-f69b-4d64-bc49-0008f0573a8e-kube-api-access-qrgtf\") pod \"authorino-bfbc85845-zqx4s\" (UID: \"89a13f94-f69b-4d64-bc49-0008f0573a8e\") " pod="kuadrant-system/authorino-bfbc85845-zqx4s" May 28 21:07:28.980161 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:28.980132 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bfbc85845-zqx4s" May 28 21:07:29.114251 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:29.114223 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bfbc85845-zqx4s"] May 28 21:07:29.116076 ip-10-0-140-94 kubenswrapper[2563]: W0528 21:07:29.116049 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89a13f94_f69b_4d64_bc49_0008f0573a8e.slice/crio-a2bbcaa19a84c9a448ce437a0d72afbae195318be20edb1f353b49348f3184fe WatchSource:0}: Error finding container a2bbcaa19a84c9a448ce437a0d72afbae195318be20edb1f353b49348f3184fe: Status 404 returned error can't find the container with id a2bbcaa19a84c9a448ce437a0d72afbae195318be20edb1f353b49348f3184fe May 28 21:07:29.465059 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:29.464977 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bfbc85845-zqx4s" event={"ID":"89a13f94-f69b-4d64-bc49-0008f0573a8e","Type":"ContainerStarted","Data":"a2bbcaa19a84c9a448ce437a0d72afbae195318be20edb1f353b49348f3184fe"} May 28 21:07:30.470379 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.470343 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bfbc85845-zqx4s" event={"ID":"89a13f94-f69b-4d64-bc49-0008f0573a8e","Type":"ContainerStarted","Data":"a560ddb502f21e1eef193b023e6362e6238d54a8dad720a37ea65eb7d5d76d23"} May 28 21:07:30.497376 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.497305 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-bfbc85845-zqx4s" podStartSLOduration=2.09602841 podStartE2EDuration="2.497286368s" podCreationTimestamp="2026-05-28 21:07:28 +0000 UTC" firstStartedPulling="2026-05-28 21:07:29.117416397 +0000 UTC m=+899.234809960" lastFinishedPulling="2026-05-28 21:07:29.518674341 +0000 UTC m=+899.636067918" observedRunningTime="2026-05-28 21:07:30.495415171 +0000 UTC m=+900.612808759" watchObservedRunningTime="2026-05-28 21:07:30.497286368 +0000 UTC m=+900.614679954" May 28 21:07:30.519439 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.519408 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86b69d559b-l244z"] May 28 21:07:30.519633 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.519607 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-86b69d559b-l244z" podUID="783c2cd8-baf8-45d3-8b64-451447ef0fab" containerName="authorino" containerID="cri-o://30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775" gracePeriod=30 May 28 21:07:30.768935 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.768911 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:07:30.859740 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.859709 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkd6b\" (UniqueName: \"kubernetes.io/projected/783c2cd8-baf8-45d3-8b64-451447ef0fab-kube-api-access-nkd6b\") pod \"783c2cd8-baf8-45d3-8b64-451447ef0fab\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " May 28 21:07:30.859935 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.859802 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/783c2cd8-baf8-45d3-8b64-451447ef0fab-tls-cert\") pod \"783c2cd8-baf8-45d3-8b64-451447ef0fab\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " May 28 21:07:30.860014 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.859951 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/783c2cd8-baf8-45d3-8b64-451447ef0fab-oidc-ca\") pod \"783c2cd8-baf8-45d3-8b64-451447ef0fab\" (UID: \"783c2cd8-baf8-45d3-8b64-451447ef0fab\") " May 28 21:07:30.861841 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.861807 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/783c2cd8-baf8-45d3-8b64-451447ef0fab-kube-api-access-nkd6b" (OuterVolumeSpecName: "kube-api-access-nkd6b") pod "783c2cd8-baf8-45d3-8b64-451447ef0fab" (UID: "783c2cd8-baf8-45d3-8b64-451447ef0fab"). InnerVolumeSpecName "kube-api-access-nkd6b". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 21:07:30.864595 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.864574 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/783c2cd8-baf8-45d3-8b64-451447ef0fab-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "783c2cd8-baf8-45d3-8b64-451447ef0fab" (UID: "783c2cd8-baf8-45d3-8b64-451447ef0fab"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 21:07:30.871288 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.871265 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/783c2cd8-baf8-45d3-8b64-451447ef0fab-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "783c2cd8-baf8-45d3-8b64-451447ef0fab" (UID: "783c2cd8-baf8-45d3-8b64-451447ef0fab"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 21:07:30.961508 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.961477 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkd6b\" (UniqueName: \"kubernetes.io/projected/783c2cd8-baf8-45d3-8b64-451447ef0fab-kube-api-access-nkd6b\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 21:07:30.961508 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.961505 2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/783c2cd8-baf8-45d3-8b64-451447ef0fab-tls-cert\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 21:07:30.961664 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:30.961523 2563 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/783c2cd8-baf8-45d3-8b64-451447ef0fab-oidc-ca\") on node \"ip-10-0-140-94.ec2.internal\" DevicePath \"\"" May 28 21:07:31.474341 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:31.474303 2563 generic.go:358] "Generic (PLEG): container finished" podID="783c2cd8-baf8-45d3-8b64-451447ef0fab" containerID="30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775" exitCode=0 May 28 21:07:31.474828 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:31.474358 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86b69d559b-l244z" May 28 21:07:31.474828 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:31.474384 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86b69d559b-l244z" event={"ID":"783c2cd8-baf8-45d3-8b64-451447ef0fab","Type":"ContainerDied","Data":"30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775"} May 28 21:07:31.474828 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:31.474420 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86b69d559b-l244z" event={"ID":"783c2cd8-baf8-45d3-8b64-451447ef0fab","Type":"ContainerDied","Data":"0d7d1fc7dc0f98e0b6f63aee9cb5bc4d34816539c336e6e1dcd463bcec0d1f0b"} May 28 21:07:31.474828 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:31.474434 2563 scope.go:117] "RemoveContainer" containerID="30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775" May 28 21:07:31.483410 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:31.483394 2563 scope.go:117] "RemoveContainer" containerID="30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775" May 28 21:07:31.483640 ip-10-0-140-94 kubenswrapper[2563]: E0528 21:07:31.483622 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775\": container with ID starting with 30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775 not found: ID does not exist" containerID="30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775" May 28 21:07:31.483682 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:31.483649 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775"} err="failed to get container status \"30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775\": rpc error: code = NotFound desc = could not find container \"30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775\": container with ID starting with 30e6dcf7a199c7c238c830879f91e2f208ce710eba30e5f819244e90c3ed0775 not found: ID does not exist" May 28 21:07:31.493998 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:31.493976 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86b69d559b-l244z"] May 28 21:07:31.501350 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:31.501331 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-86b69d559b-l244z"] May 28 21:07:32.469032 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:07:32.468959 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="783c2cd8-baf8-45d3-8b64-451447ef0fab" path="/var/lib/kubelet/pods/783c2cd8-baf8-45d3-8b64-451447ef0fab/volumes" May 28 21:30:06.660779 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:06.660744 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-bfbc85845-zqx4s_89a13f94-f69b-4d64-bc49-0008f0573a8e/authorino/0.log" May 28 21:30:11.834313 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:11.834281 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc97fc4b4-gpspq_d9792bf4-afb0-42af-b33d-4cb081565ea1/manager/0.log" May 28 21:30:13.311008 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:13.310980 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-bfbc85845-zqx4s_89a13f94-f69b-4d64-bc49-0008f0573a8e/authorino/0.log" May 28 21:30:14.095176 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:14.095130 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-g6mjz_c8941719-cbf5-42d7-a7c1-0f2dc49e456a/manager/0.log" May 28 21:30:14.643223 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:14.643192 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-774cfddf4d-2hcfs_af2b87fd-53fb-4a13-8667-c2e686fd56f6/kube-auth-proxy/0.log" May 28 21:30:15.088485 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:15.088431 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b5d5f9b5b-76hl9_f9e058d4-ea36-4595-8372-5f2839212a4b/router/0.log" May 28 21:30:15.417416 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:15.417344 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj_511a4a15-b5eb-45c7-8271-a22f974fa52a/storage-initializer/0.log" May 28 21:30:15.423295 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:15.423268 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-jd2vj_511a4a15-b5eb-45c7-8271-a22f974fa52a/main/0.log" May 28 21:30:15.526346 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:15.526319 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2_ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c/storage-initializer/0.log" May 28 21:30:15.532954 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:15.532934 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-ndvk2_ff4cf9e0-4853-4e3d-bedc-f5cb3d8acb9c/main/0.log" May 28 21:30:15.637693 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:15.637668 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-rclhl_4dcddbcc-7ac7-45eb-a225-f2239d5fd28a/storage-initializer/0.log" May 28 21:30:15.644444 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:15.644427 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-rclhl_4dcddbcc-7ac7-45eb-a225-f2239d5fd28a/main/0.log" May 28 21:30:22.527045 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:22.527014 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xsvnt_15cde4c4-5e59-47e6-87f4-32de185a998e/global-pull-secret-syncer/0.log" May 28 21:30:22.572541 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:22.572512 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2nxmw_b289084f-e171-435e-8cf2-d5ba9965c060/konnectivity-agent/0.log" May 28 21:30:22.692834 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:22.692804 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-94.ec2.internal_be9633c9382b980a365833a255350af4/haproxy/0.log" May 28 21:30:26.827213 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:26.827160 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-bfbc85845-zqx4s_89a13f94-f69b-4d64-bc49-0008f0573a8e/authorino/0.log" May 28 21:30:27.098794 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:27.098704 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-g6mjz_c8941719-cbf5-42d7-a7c1-0f2dc49e456a/manager/0.log" May 28 21:30:28.214617 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.214590 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28c0c283-4236-4cb1-a514-4ff3f334c8ef/alertmanager/0.log" May 28 21:30:28.243818 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.243794 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28c0c283-4236-4cb1-a514-4ff3f334c8ef/config-reloader/0.log" May 28 21:30:28.281240 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.281204 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28c0c283-4236-4cb1-a514-4ff3f334c8ef/kube-rbac-proxy-web/0.log" May 28 21:30:28.305790 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.305768 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28c0c283-4236-4cb1-a514-4ff3f334c8ef/kube-rbac-proxy/0.log" May 28 21:30:28.332163 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.332142 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28c0c283-4236-4cb1-a514-4ff3f334c8ef/kube-rbac-proxy-metric/0.log" May 28 21:30:28.354038 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.354017 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28c0c283-4236-4cb1-a514-4ff3f334c8ef/prom-label-proxy/0.log" May 28 21:30:28.376052 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.376007 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_28c0c283-4236-4cb1-a514-4ff3f334c8ef/init-config-reloader/0.log" May 28 21:30:28.565040 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.565017 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-qgkh5_c2c83195-81e1-4d41-a670-5e3e9523f78f/monitoring-plugin/0.log" May 28 21:30:28.796760 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.796733 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tnmmc_c7dd10c3-795a-4a25-b7f4-6e5be725dc6f/node-exporter/0.log" May 28 21:30:28.826069 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.826014 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tnmmc_c7dd10c3-795a-4a25-b7f4-6e5be725dc6f/kube-rbac-proxy/0.log" May 28 21:30:28.848203 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.848184 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tnmmc_c7dd10c3-795a-4a25-b7f4-6e5be725dc6f/init-textfile/0.log" May 28 21:30:28.985176 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:28.985141 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a2ec453-3aea-4cb2-971f-b962455fe573/prometheus/0.log" May 28 21:30:29.012384 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:29.012356 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a2ec453-3aea-4cb2-971f-b962455fe573/config-reloader/0.log" May 28 21:30:29.037931 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:29.037913 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a2ec453-3aea-4cb2-971f-b962455fe573/thanos-sidecar/0.log" May 28 21:30:29.060927 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:29.060909 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a2ec453-3aea-4cb2-971f-b962455fe573/kube-rbac-proxy-web/0.log" May 28 21:30:29.084958 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:29.084894 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a2ec453-3aea-4cb2-971f-b962455fe573/kube-rbac-proxy/0.log" May 28 21:30:29.107205 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:29.107186 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a2ec453-3aea-4cb2-971f-b962455fe573/kube-rbac-proxy-thanos/0.log" May 28 21:30:29.128875 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:29.128854 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a2ec453-3aea-4cb2-971f-b962455fe573/init-config-reloader/0.log" May 28 21:30:29.169062 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:29.169038 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-6cbdb_09bff3df-cdf6-4388-85c9-400f4cfc9264/prometheus-operator/0.log" May 28 21:30:29.189909 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:29.189888 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7779bbdd78-6cbdb_09bff3df-cdf6-4388-85c9-400f4cfc9264/kube-rbac-proxy/0.log" May 28 21:30:29.215710 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:29.215680 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-9tbqw_0bd2d4c1-7b03-4fa9-ada3-4403d1778e4a/prometheus-operator-admission-webhook/0.log" May 28 21:30:31.085770 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.085734 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd"] May 28 21:30:31.086303 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.086285 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="783c2cd8-baf8-45d3-8b64-451447ef0fab" containerName="authorino" May 28 21:30:31.086387 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.086306 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="783c2cd8-baf8-45d3-8b64-451447ef0fab" containerName="authorino" May 28 21:30:31.086446 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.086420 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="783c2cd8-baf8-45d3-8b64-451447ef0fab" containerName="authorino" May 28 21:30:31.089994 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.089974 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.091717 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.091689 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hjxh6\"/\"default-dockercfg-dbw98\"" May 28 21:30:31.091830 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.091770 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hjxh6\"/\"kube-root-ca.crt\"" May 28 21:30:31.092236 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.092220 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hjxh6\"/\"openshift-service-ca.crt\"" May 28 21:30:31.094762 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.094738 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd"] May 28 21:30:31.179196 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.179140 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-lib-modules\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.179365 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.179205 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nss6z\" (UniqueName: \"kubernetes.io/projected/5625db06-5331-48a9-87d4-2be65b6864f5-kube-api-access-nss6z\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.179365 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.179282 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-sys\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.179365 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.179334 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-podres\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.179365 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.179362 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-proc\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.280311 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.280279 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-podres\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.280445 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.280318 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-proc\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.280445 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.280358 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-lib-modules\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.280445 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.280376 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nss6z\" (UniqueName: \"kubernetes.io/projected/5625db06-5331-48a9-87d4-2be65b6864f5-kube-api-access-nss6z\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.280445 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.280424 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-sys\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.280659 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.280453 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-podres\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.280659 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.280454 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-proc\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.280659 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.280537 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-sys\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.280659 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.280537 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5625db06-5331-48a9-87d4-2be65b6864f5-lib-modules\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.287512 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.287489 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nss6z\" (UniqueName: \"kubernetes.io/projected/5625db06-5331-48a9-87d4-2be65b6864f5-kube-api-access-nss6z\") pod \"perf-node-gather-daemonset-rlsfd\" (UID: \"5625db06-5331-48a9-87d4-2be65b6864f5\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.400620 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.400538 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:31.526218 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.526190 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd"] May 28 21:30:31.532121 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.532098 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 21:30:31.717417 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:31.717355 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-dzhkc_1c261638-eb90-4ed1-8125-2210a314d8d8/download-server/0.log" May 28 21:30:32.208036 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:32.208011 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-ks28m_81a8812c-d4bf-4281-8cca-7bde3ceb0dc3/volume-data-source-validator/0.log" May 28 21:30:32.212699 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:32.212672 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" event={"ID":"5625db06-5331-48a9-87d4-2be65b6864f5","Type":"ContainerStarted","Data":"7e5806518afb89e2d92e3e54c3ac33d8ed8d1fdae794ac5fc94939037ec49579"} May 28 21:30:32.212699 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:32.212707 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" event={"ID":"5625db06-5331-48a9-87d4-2be65b6864f5","Type":"ContainerStarted","Data":"ab63ada98703335f4b56d2b0adec948911f89cdb1b971f5b31a0525adeff8cd0"} May 28 21:30:32.212891 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:32.212726 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:32.226881 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:32.226840 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" podStartSLOduration=1.226827149 podStartE2EDuration="1.226827149s" podCreationTimestamp="2026-05-28 21:30:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 21:30:32.224848603 +0000 UTC m=+2282.342242187" watchObservedRunningTime="2026-05-28 21:30:32.226827149 +0000 UTC m=+2282.344220734" May 28 21:30:33.129581 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:33.129559 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h7jdr_c81ee9a4-8670-4a9f-8326-e30cf38d0ee9/dns/0.log" May 28 21:30:33.149979 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:33.149954 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h7jdr_c81ee9a4-8670-4a9f-8326-e30cf38d0ee9/kube-rbac-proxy/0.log" May 28 21:30:33.194064 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:33.194043 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lkcmf_946679ad-8718-48c0-98c2-03feb3f67b30/dns-node-resolver/0.log" May 28 21:30:33.712336 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:33.712306 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9lkdc_943ced3a-27aa-4fb2-8ffd-53d1269faf80/node-ca/0.log" May 28 21:30:34.717741 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:34.717710 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-774cfddf4d-2hcfs_af2b87fd-53fb-4a13-8667-c2e686fd56f6/kube-auth-proxy/0.log" May 28 21:30:34.894599 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:34.894571 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b5d5f9b5b-76hl9_f9e058d4-ea36-4595-8372-5f2839212a4b/router/0.log" May 28 21:30:35.404911 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:35.404885 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s64pw_87bd9d8d-23ac-400a-8c2f-b368202b054e/serve-healthcheck-canary/0.log" May 28 21:30:35.926558 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:35.926528 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dpgcb_f5d326dd-b9d0-4249-a11e-fb2f4829b875/kube-rbac-proxy/0.log" May 28 21:30:35.947730 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:35.947707 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dpgcb_f5d326dd-b9d0-4249-a11e-fb2f4829b875/exporter/0.log" May 28 21:30:35.969410 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:35.969384 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dpgcb_f5d326dd-b9d0-4249-a11e-fb2f4829b875/extractor/0.log" May 28 21:30:38.159493 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:38.159468 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc97fc4b4-gpspq_d9792bf4-afb0-42af-b33d-4cb081565ea1/manager/0.log" May 28 21:30:38.225203 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:38.225156 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-rlsfd" May 28 21:30:39.428210 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:39.428163 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8459dbdd59-lxd2q_ab3caa90-b7f3-4037-9065-cebaf9b6f1c4/manager/0.log" May 28 21:30:43.852863 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:43.852838 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-fndzz_f46c90a9-c203-4962-acd5-5fe69b70180b/migrator/0.log" May 28 21:30:43.875077 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:43.875050 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-fndzz_f46c90a9-c203-4962-acd5-5fe69b70180b/graceful-termination/0.log" May 28 21:30:44.238530 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:44.238502 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-hrsg6_d4caaa6c-18e0-4e2b-a12c-1f596369b74a/kube-storage-version-migrator-operator/1.log" May 28 21:30:44.239413 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:44.239384 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-hrsg6_d4caaa6c-18e0-4e2b-a12c-1f596369b74a/kube-storage-version-migrator-operator/0.log" May 28 21:30:45.204720 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:45.204689 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4tz8l_aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0/kube-multus-additional-cni-plugins/0.log" May 28 21:30:45.227479 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:45.227456 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4tz8l_aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0/egress-router-binary-copy/0.log" May 28 21:30:45.246955 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:45.246933 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4tz8l_aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0/cni-plugins/0.log" May 28 21:30:45.267762 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:45.267741 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4tz8l_aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0/bond-cni-plugin/0.log" May 28 21:30:45.287942 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:45.287922 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4tz8l_aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0/routeoverride-cni/0.log" May 28 21:30:45.309696 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:45.309679 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4tz8l_aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0/whereabouts-cni-bincopy/0.log" May 28 21:30:45.330626 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:45.330608 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4tz8l_aa3c1cc6-50e9-45e4-aa35-5028cc0c85a0/whereabouts-cni/0.log" May 28 21:30:45.726183 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:45.726152 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-l89rg_9ac6fddb-e201-4648-8670-9a00e75dd19f/kube-multus/0.log" May 28 21:30:45.841785 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:45.841758 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qqths_338d9053-7caa-48f3-bd43-6433cf054fc5/network-metrics-daemon/0.log" May 28 21:30:45.860558 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:45.860536 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qqths_338d9053-7caa-48f3-bd43-6433cf054fc5/kube-rbac-proxy/0.log" May 28 21:30:47.243527 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:47.243498 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nlv9g_e6f5ed42-ee6c-4a50-a715-93c3b4e92da8/ovn-controller/0.log" May 28 21:30:47.272621 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:47.272597 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nlv9g_e6f5ed42-ee6c-4a50-a715-93c3b4e92da8/ovn-acl-logging/0.log" May 28 21:30:47.291293 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:47.291271 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nlv9g_e6f5ed42-ee6c-4a50-a715-93c3b4e92da8/kube-rbac-proxy-node/0.log" May 28 21:30:47.312599 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:47.312571 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nlv9g_e6f5ed42-ee6c-4a50-a715-93c3b4e92da8/kube-rbac-proxy-ovn-metrics/0.log" May 28 21:30:47.334129 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:47.334109 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nlv9g_e6f5ed42-ee6c-4a50-a715-93c3b4e92da8/northd/0.log" May 28 21:30:47.355108 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:47.355086 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nlv9g_e6f5ed42-ee6c-4a50-a715-93c3b4e92da8/nbdb/0.log" May 28 21:30:47.377492 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:47.377463 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nlv9g_e6f5ed42-ee6c-4a50-a715-93c3b4e92da8/sbdb/0.log" May 28 21:30:47.481296 ip-10-0-140-94 kubenswrapper[2563]: I0528 21:30:47.481257 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nlv9g_e6f5ed42-ee6c-4a50-a715-93c3b4e92da8/ovnkube-controller/0.log"